ReST DataSource plugin for CakePHP
A CakePHP Plugin containing a DataSource for interarcting with ReSTful web services.
It’s becoming increasingly common to integrate information and services from 3rd parties into our applications via web services. Technologies allowing you to do this include SOAP and XML-RPC but ReST APIs are becoming the predominant method for interacting with them, probably because ReST is really simple.
I’ve been working on a lot projects recently that consume ReStful web services from several 3rd parties including:
(Stay tuned for source code and blog posts on CakePHP plugins for these APIs)
I’ve been implementing code to access these APIs as plugins with their own data sources, and noticed that I was writing the same code over and over, so I decided to abstract the common functionality, the stuff that was API agnostic, the stuff that purely dealt with issuing requests and dealing with responses into a ReST data source, that my other API specific data sources extend. (The current implementation is actually based on a RestSource class I originally bundled with my gdata plugin, but it’s had a few tweaks and is now available in it’s own plugin.)
You can get the code here.
How it works
- The DataSource issues HTTP requests through CakePHP’s HttpSocket class (default) or your own implementation/extension, e.g. an OAuth enabled HttpSocket http://github.com/neilcrookes/http_socket_oauth, passed in in the constructor.
- It implements create, read, update and delete methods, which are called by Model::save(), Model::find() and Model::delete(). These methods expect your Model object to have a request property in the format defined in HttpSocket::request and add the appropriate HTTP verbs into the array if not already set, e.g. POST, GET, PUT, DELETE.
- These methods call the public RestSource::request() method passing it an instance of your model object, but you can also access it directly, passing an object with a request property, or an arrayin the format defined in HttpSocket::request or a string which is a URI.
- The raw response from the HttpSocket::request() is converted into an array, according to the response’s content-type header (XML and JSON currently supported), and the result returned on success (determined by 2xx HTTP response code) or boolean false is returned on failure.
- In addition, if the argument to the request() method was an object, the result is also added to a response property of the object, and if the request failed, and the object has an onError() method, that method will be triggered.
Direct Usage
- Create a model for each thing on web service you want to interact with. For example create a TwitterStatus model
- Ensure your model uses this datasource by setting it in the $useDbConfig property, e.g.
public $useDbConfig = 'Rest.Rest'; - Create a method in you model that corresponds to a function you can perform through the web service. E.g.
public function save($data = null, $validate = true, $fieldList = array()) {
$this->request = array(
'uri' => array(
'host' => 'twitter.com',
'path' => 'statuses/update.json'
),
'body' => array(
'status' => $data['TwitterStatus']['text']
)
);
return parent::save($data, $validate, $fieldList);
}
- Anywhere in your application, call:
$result = ClassRegistry::init('TwitterStatus')->save(array('TwitterStatus' => array('text' => 'Hello World!')));
N.B. This example is purely representative and does not include authentication for example, but demonstrate the general idea.
Other uses
As I mentioned above, I abstracted this functionality out of a lot of other datasources I was writing and I use this extensively as a base data source which I extend for each web service I need, handling the specifics of each of those web services, such as Authentication, in those classes, by overloading each of the public methods in the Rest datasource as required.
For example, you can create a TwitterSource which extends RestSource and overload the constructor to pass in an instance of my HttpSocketOauth extension to CakePHP’s HttpSocket class, and the request() method, adding in the common request params such as the host key and ‘.json’ on the end of the path key, and the specific authentication params, then just call parent::request() to handle the issuing of the request and parsing of the response. (As I said above, blog post coming soon about the Twitter plugin and others).
(5 votes, average: 4.40 out of 5)
Loading ...
19 Responses so far
June 2nd, 2010
1:21 pm
I am using cake 1.3.1, and want to use this plugin. So I bake the Rest plugin to make sure it has all the necessary folders (probably unnecessary):
>>cake bake plugin rest
Then I copy your model directory to:
/app/plugin/rest
And add this to my ‘/app/config/database.php’, (maybe you should add this step to the instructions above, or am I missing something).
var $rest = array(
'datasource' => 'Rest.Rest',
);
Then do a similar example to the one you showed:
public $useDbConfig = 'rest'; //Because that config has to be in my
//database.php file.
Cake does not find the datasource. If I copy the datasource into the ‘/app/models/datasources’ directory and change the database.php file to
var $rest = array(
'datasource' => 'Rest',
);
it works. I know that there have been some issues with plugin routing in 1.3, do you know if this is common problem? (I cleared my routes.php file to make sure it is not causing the problem and the cache is off and cloned your code today.)
June 3rd, 2010
12:43 pm
Hi Adrian,
Thanks for your comment.
Good point about adding the config to database.php
I’ve always used this datasource for models in a plugin, so haven’t tried it this way, even though, it is probably the most common way other people will use it.
The cook book says add the following to your database.php
var $lastFm = array(
'datasource' => 'WebservicePack.LastFm'
...
So we’d do…
var $rest = array(
'datasource' => 'Rest.Rest'
...
Then I guess we’d specify the dbconfig in the model as you have done:
public $useDbConfig = 'rest';
June 9th, 2010
3:54 pm
Hi,
I am using cake 1.3.1 too, and works fine with following configuration.
1. Make directory ‘APP/plugins/rest/models/datasources’ and copy ‘rest_source.php’ into it.
2. Edit APP/config/database.php as following:
class DATABASE_CONFIG {
var $rest = array(
'datasource' => 'Rest.RestSource',
);
}
3. To use REST datasource in model:
// APP/model/foo.php
class Foo extends AppModel {
var $useTable = false;
var $useDbConfig = 'rest';
...
I hope this helps.
September 17th, 2010
10:06 pm
Neil,
Awesome job with this datasource, but just one question:
How does it not run into problems with the delete() method? When you delete, Cake attempts to perform a bunch of logic, including calling the datasource’s calculate() method, doing a count, etc.
September 17th, 2010
11:56 pm
@Anthony. Thanks. Can’t remember what happened when I tried this. Can you provide a calculate() method in your DataSource?
September 18th, 2010
12:37 am
You can, but it doesn’t really make sense to use with a REST-ful source. When you try to delete something in Cake, it automatically checks for the existence of the record using find(‘count’). Before it does that, it uses DataSource::calculate() to figure out the name of the count “field”… none of this makes sense as it relates to REST. It seems as though you’d need to overwrite Model::delete() so that it doesn’t attempt to perform all this logic, but I don’t see you doing that. So I’m confused as to how this whole structure works
September 18th, 2010
10:11 am
@Anthony, I do in fact overwrite Model::delete() in some of my plugins that implement delete functionality. See TwitterStatus::delete()
September 18th, 2010
10:15 am
@Anthony, if you’re still confused about the structure/approach, have a look at the presentation I gave at CakeFest a couple of weeks ago. Also, see this diagram. If you’re still confused, let me know and I’ll get out another blog post that eplains it all a bit better. Thanks.
September 18th, 2010
3:50 pm
Neil,
I was actually at CakeFest and saw your presentation. It’s what inspired me to give this a shot :)
So anyway – I tried out your Twitter plugin, and it turns out that it suffers from the exact issue I was referring to:
If I try going to:
http://myapplication.com/twitter/twitter_statuses/delete/some_id_i_just_created
… I get:
Fatal error: Call to undefined method TwitterSource::calculate() in X:\myapplication\cake\libs\model\model.php on line 2138
I also have issues with the plugin trying to create a ‘twitter’ datasource even though I have one defined in database.php. Apparently, ConnectionManager::sourceList() isn’t actually outputting the list of sources that are available for whatever reason, but if I simply delete that functionality and use ConnectionManager::getDataSource(‘twitter’), assuming it’s there (because it is), then everything works just fine.
Anyway – I wonder if I’m doing something wrong, or if there is an inconsistency in Cake versions, or if maybe this code just was never tested for the cases I’m describing? Let me know if you have any insight
September 19th, 2010
11:38 pm
@Anthony, I’ll look into the issues you describe and get back to you. The Twitter plugin is not finished, there may be a couple of bugs, such as the one with TwitterStatus::delete(). Can you confirm what version of Cake you’re running?
September 20th, 2010
1:06 am
I believe it’s 1.3.2.
I think this delete() issue is a problem in general. It should affect *any* logic that is in any way calling the parent Model::delete(), not just the Twitter plugin
September 24th, 2010
7:15 am
I am working with your plugin and all is going well except to the point where I am trying to submit a twitter status update. I KNOW ABOUT the oAuth thing and since I am only tweeting to our one account, I have retrieved the information that twittter says is required from the developer system off my app. I have put it in and the only response when an update is issued is “1″. Is there a way to turn on more debugging or something that can tell me where the breakdown is?
Thanks!
Andrew
November 22nd, 2010
10:13 pm
Hey – this looks great; I’m totally going to use it.
I noticed something for the next version, though – in RestSource.request(), you should add a case for ‘text/xml’, which is a content type I’ve encountered for xml. I made the change in my own file, but other people might find this helpful, as well …
November 23rd, 2010
10:22 pm
@Peter, thanks, will do
November 23rd, 2010
10:28 pm
@Andrew, sorry for not replying sooner, you’ll have sorted it by now I’m sure, but for the benefit of anyone else, you can access the raw http response. In your model use something like:
$this->getDataSource()->response['raw']
January 5th, 2011
6:59 am
Very interesting find. I read the slides and got a bad feeling in my stomach. I later realized it was regret from not partaking in the awesome-ness that was CakeFest 2010. At any rate, I came across this post because I too have been looking for ways to make use of the build in model features for my Amazon datasource. I wanted to be able to do stuff like $this->Model->find(‘first) and be able to trigger afterFind methods. I am stumped with DataSource:calculate() as I have no clue what cake expects me to return. I created the function and threw in debug(func_get_args()) but all i see is the model object and a string ‘count’ (this is when i do $this->Model->find(‘count’). I assume once i figure this out i wont have to use my custom paginateCount and paginate methods in my model.
Another thing, crating models for specific entities to be accessed in your API is simply brilliant. I am in the process of creating a Books model where I would define properties and methods specific to accessing Books from the amazon API. You think the Twitter api is massive? The Amazon API docs put the Twitter API to shame! lol. By having a Book model that interacts with my Amazon datasource I can define a “normalize” function that, after it gets the crazy array from my datasource’s read method, can modify the array to match what a find() would normally return.
Keep up the great work Neil and see you at CakeFest 2011!
January 21st, 2011
8:12 pm
Hi everyone,
I have set up this rest datasource, so now I need to place some functions which are to:
1)Send data to the Rest API
2)Receive data from the Rest API
May I know where do I place the dynamic variables such as, userid, sessionid, itemid so that it can be sent to to the API in the Send function ?
The reason it is dynamic is because, each user view a page, e.g. view/5 by userid=1, item=5 these data are sent to the rest API when they view that page. I wonder where do these functions should be stored? I have the working piece of codes in a non-cake PHP scripting, I just can’t figure out where to place it in, e.g. in my Rest Model? the Controller which I want to use to display the data? or in the Rest datasource?
Hope someone could clarify this for a bit. Thanks.
regards,
John Maxim
January 22nd, 2011
7:40 am
Hi,
Updating my question so it better presents my problem:
I have set up this rest datasource, so now I need to place some
functions which are to:
1)Send data to the Rest API
2)Receive data from the Rest API
May I know where do I place the dynamic string variables such as,
userid, sessionid, itemid so that it can be sent to to the API which
is currently in the “Send function” ?
The reason it is dynamic is because, each user view a page, e.g. view/
5 by userid=1, item=5: these data are sent to the rest API when they
view that page. I wonder where do these dynamic string variables
functions should be stored? I have the working piece of codes in a non-
cake PHP scripting, I just can’t figure out where to place it in, e.g.
in my Rest Model? the Controller which I want to use to display the
data? or in the Rest datasource?
Here’s what I mean:
The member’s data:(E.g.)
$itemId = “5″;
$sessionid = “F3D4E3BE31EE3FA069F5434DB7EC2E34″;
$itemdescription = “Chicken Chop”;
$itemurl = “/food/view/5″;
$itemImageurl = “*/sample-image-url.jpg”;
$userid = “24EH17KOMEWKKJNS89″;
$timeRange = “ALL”;
Send Action:
$eventReply = sendAction(“view”, $itemId, $itemdescription ,$itemurl ,
$itemimageurl,$sessionid,$userid);
….
… //and other functions
…
…
I have checked out other developers sending to ReST but mostly using
static data. I’m not sure though. (e.g.) Google Analytics, Yahoo or
Flickr, Twitter, which usually allow us to CRUD. But what about
dynamically sending and receiving data ? I’m quite sure it can be done
but nothing much can be read about this.
Hope someone could clarify this for a bit. Thanks.
regards,
John Maxim
July 23rd, 2011
3:58 pm
[...] Neil Crookes » ReST DataSource plugin for CakePHP. [...]
Leave a comment