In the past, delivering content to mobile devices has been a very tricky subject. Developers who came into the mobile world were usually confronted with a new and unknown paradigm, where very little information could be found on how to determine devices’ capabilities and to deliver content to them. It was something completely new, and it looked like this information was kept secretly as a precious treasure by those few who had been able to learn it. The aim of this article is to introduce practical examples on how to deliver content to Mobile Devices, making use of sample source code where possible.
In the past, developers had to rely on the very little information that was available on the Internet, and on their own tests via trial and error. Furthermore, the lack of standarisation among phone manufacturers made things even worse, since each of them supported different file formats, and different ways of downloading content, and so on…
As time went by, information slowly started becoming available, communities of Mobile Developers began to get together on mailing-lists, and great utilities like WURFL and others came to save us.
Fortunately, each day more resources for Mobile Developers continue to appear. Great examples of this are mobiForge and DeviceAtlas.
But despite this, we still often lack clear examples on how to achieve certain tasks, or solve particular problems.
So, in this article we present some practical examples on how to deliver content to Mobile Devices, and make use of sample source code where possible.
Use of DeviceAtlas and others goes beyond the scope of this article, but it is recommended to check out the DeviceAtlas tutorial.
Code examples are in PHP, but should easily be ported to other programming languages.
Serving content
Over the years different mobile devices have supported different ways of downloading content, some of them which are today almost deprecated; examples of these include Openwave’s Download Fun or Nokia’s COD.
Nowadays, practically all modern mobile devices support what we call “direct download“, so this is the method I am going to describe in this article, plus some variations which make use of download descriptor files.
There are two main ways of serving files for content delivery:
- Serving real files on the server’s filesystem
- Using a script or servlet which streams the content
Serving real files on the server’s filesystem
Serving real files is probably the most “safe” way of delivering content. It saves the developer from having to mess with Mobile Browsers’ different implementations (which may well behave in different ways), by leaving that burden to the Webserver (Apache, IIS, or other). Webservers already know how to do this job, taking care of handling the different types of HTTP requests, and giving back the correct responses on each case.
The “drawback” with this method is that most of the time we will not want to have the content accessible to the whole world. In most cases we will be billing the customer for the content he’s downloading, so we don’t want other people to be able to freely download it. The idea then, is to store the content outside of the Web server’s document root, and when requested, copy it to some temporary location for download. Some time after the customer has downloaded the content, we erase it from its temporary location.
No matter what way the customer buys the content, we will usually end up sending him a WAP Push or a text message with a URL from where the content may be downloaded (MMS may be another option, but it’s also out of the scope of this article). A recommended way of doing this, is by using URLs like:
http://wap.mydomain.tld/get.php/123456abcdef
On this example URL, 123456abcdef is a unique ID, which corresponds to our customer’s specific request. When the customer requests some content (by sending a text message to some short code, or whatever other method) we insert a record in our database with a unique ID which identifies that request, and indicates what content should be delivered. Then we send him the URL with this unique ID, so when he connects to it, we look for the record which is associated with this ID and can determine what content to serve.
Below is some sample code that could be used for the get.php script (the script to match the URL with the content):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
<?php $request_array = explode('/', $_SERVER['REQUEST_URI']); $unique_id = $request_array[2]; $dbc = mysql_connect('myHost', 'myUsername', 'myPasswd') or die('Error connecting to MySQL server'); mysql_select_db('myDB', $dbc); $sql = "SELECT contentId FROM downloadRequests WHERE uniqueID = '$unique_id'"; $result = mysql_query($sql, $dbc); if(mysql_num_rows($result) > 0) { list($content_id) = mysql_fetch_row($result); $filename = writeTmpContent($content_id); $download_url = 'http://'.$_SERVER['SERVER_NAME'].'/tmp/'.$unique_id.'/'.$filename; displayDownloadPage($download_url); } else { // display some error page } mysql_close($dbc); ?> |
In this sample code we are first obtaining the ID of the content the customer has requested, by using the unique ID from the URL. Then we call the writeTmpContent() function, passing to it the content ID as a parameter. This function should check what file formats and capabilities the device supports (e.g. by using DeviceAtlas, WURFL, or other), and write the correct file to a temporary location under the Web server’s document root. On our example we create a directory named like the URL’s unique ID under a “tmp” directory. The function returns the filename, which would be something like “image.jpg” or “ringtone.mp3“. We then call the displayDownloadPage() function which will display a page to the customer saying something like “Click here to download your content“. This text would contain a link to $download_url. Of course, this is just an example, and it assumes a mysql database with a particular schema. You should adapt the code to your own DB schema and needs.
It is highly recommended to use WALL or some other content adaptation engine to display the download page, in order to be able to render the most appropriate markup language for each device, without having to manually implement it.
You may be tempted to use an HTTP redirect to the content file, instead of using this intermediate download page, but it is not recommended to do so, since not all devices support HTTP redirects. Also this download page can give a nice user experience, by placing your company’s logo on it, or you could also use it for supplying some supplementatry information or advertisement.
After the user has downloaded the content, we must erase this temporary file, and deactivate the unique ID so that it is not used again.
There may be several approaches to this:
- One approach is to mark the file on our database as having been downloaded as soon as the customer accesses the
get.phpscript. Then have a cronjob that runs every X minutes, which searches our database for newly downloaded files, and erases them. There is a drawback with this, which is that the user may have not been able to download the content after accessing theget.phpscript, because of connection problems or other, and our cronjob will erase the temporary file, despite of not having been really downloaded. It is recommended then, to not erase these files before some reasonable time has passed. - Other approaches can be to check if the Web server has sent all the file’s data to the client (left as an exercise to the reader) or use OMA Download, which will be described later on this article.
- Some carriers sometimes require that a file must be able to be downloaded x times by the customer, so that the user can retry the download several times in case it does not download completely on some attempt.
It may be a good idea to register the User-Agent of the device which has downloaded the content, so on further attempts you can check if it’s the same User-Agent. If it’s not the same User-Agent, it probably means that someone else is trying to download the content, and you probably won’t deliver the content in this case (and display some error message to the user).
Note that the appropriate MIME-Types for each file type to be served must be configured on the Webserver.
Some file types have different variations of MIME-Types that can be used.
From some of the most common file types, these are the MIME-Types I have found to be the most compatible with mobile devices:
| File extension | MIME-Type |
|---|---|
jpg |
image/jpeg |
gif |
image/gif |
png |
image/png |
mid |
audio/midi |
amr |
audio/amr |
mmf |
application/vnd.smaf |
mp3 |
audio/mpeg |
qcp |
audio/vnd.qcelp |
jad |
text/vnd.sun.j2me.app-descriptor |
jar |
application/java-archive |
3gp |
video/3gpp |
3g2 |
video/3gpp2 |
Using a script which streams the content
This approach is easier from the point of view that you will be streaming the content to the device, so you don’t need to create temporary files, nor run some cronjob which later erases them.
The problem with this approach is that you have to program all the HTTP headers manually; mainly the ones from your response, but in some cases it may be also necessary to handle the client’s request headers also.
By testing different combinations of various recommendations to take into account when streaming content, (posted years ago on the wmlprogramming mailing-list) I have found the best way of doing it by:
- Sending only the
Content-TypeandFile-LengthHTTP headers - Including the filename on the end of the URL (like “
image.jpg” or “ringtone.mp3“)
It is important not to use the “Content-Disposition” HTTP header, since some phones refuse to accept content when using it.
By including the filename on the URL, you will trick the phone to think it’s a real file and to accept it.
Now, when you send the download URL to the customer, you don’t normally know yet what device the customer has, so you don’t know what file formats the device will support. Therefore, you can’t include the filename on that URL, and once again, you will need an intermediate download page. Once more, we will use a URL like:
http://wap.mydomain.tld/get.php/123456abcdef
This time, when the customer connects to download the content, the get.php script will not create a temporary file, but point to another script which streams the file contents.
Supposing the resultant content to download will be “image.jpg“, the intermediate download page could point the customer to a URL like:
http://wap.mydomain.tld/download.php/123456abcdef/image.jpg
The relevant part of code on the download.php script could be something like the following:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<?php // first get the path to the content and it's MIME-Type, based on the URL's unique ID if (is_file($path_to_content)) { header("Content-Type: ".$mime_type); header("Content-Length: ".filesize($path_to_content)); readfile($path_to_content); } else { // some error... } ?> |
Note that this approach doesn’t free us from the fact that the user may have connectivity problems (or other) while downloading the content.
This method has been found to work really well, and no device has yet been detected for which it doesn’t work, except for Apple’s iPhone (when streaming audio and video).
SeeAppendix A for sample code on how to make it work on iPhone.
OMA Download
As we have previously mentioned, we normally don’t have much way of knowing if a customer effectively completed to download his content, or if the content was successfully installed/rendered on his device. This is where OMA Download comes into play.
OMA Download, many times referred as OMA DD, is an open standard and application-level protocol that enables reliable content downloads.
It provides the ability for devices to check if the content will be able to be downloadable and usable on the device, and the ability for providers to receive a confirmation of the download status.
It’s supported by most modern devices, but you should check on each download if the device supports it, in order to use it or not before using this approach (determine using DeviceAtlas, WURFL, header checking or other).
How it works
OMA Download makes use of a Download Descriptor (that’s where the “DD” in OMA DD comes from), which is a simple XML file containing some specific elements. Instead of directly downloading the content, the customer will be pointed to download this descriptor file, which will contain all the relevant information for the device to be able to download the content.
The following list describes the possible elements to be included on the descriptor:
| Name | Definition |
|---|---|
type |
The MIME media type of the media object |
size |
The number of bytes to be downloaded from the URI |
objectURI |
The URI (usually URL) from which the media object can be loaded |
installNotifyURI |
The URI (or usually URL) to which a installation status report is to be sent, either in case of a successful completion of the download, or in case of a failure |
nextURL |
The URL to which the client should navigate in case the end user selects to invoke a browsing action after the download transaction has completed with either a success or a failure |
DDVersion |
The version of the Download Descriptor technology |
name |
A user readable name of the Media Object that identifies the object to the user |
description |
A short textual description of the media object |
vendor |
The organisation that provides the media object |
infoURL |
A URL for further describing the media object |
iconURI |
The URI of an icon |
installParam |
An installation parameter associated with the downloaded media object |
Actually, only the first 3 elements (type, size and objectURI) are mandatory on the Download Descriptor.
The installNotifyURI element is not mandatory on the descriptor definition, but it probably does not make much sense on using OMA Download if you’re not going to define this element. This is the URI (usually URL) where the device will post the download status to, and it’s the main reason of the creation of this standard.
name and description are very important and useful parameters. When a device downloads the Download Descriptor, it’s mobile browser will display some of the descriptor’s data, like the file size, and in most cases the name and description. Despite the fact that these two elements are not mandatory on the browser implementation, they are usually implemented since they help the user understand what he’s downloading.
nextURL can be very useful too (but is also optional for the browser implementation). Once the download is completed, the browser will be automatically redirected to the specified URL. Some content providers might use it to up-sell more content, or you might want to redirect to instructions on how to use the downloaded content or to some other resource.
This is the content of an example Download Descriptor (taken from the OMA Download specification):
|
1 2 3 4 5 6 7 8 |
<media xmlns="http://www.openmobilealliance.org/xmlns/dd"> <type>image/gif</type> <objectURI>http://download.example.com/image.gif</objectURI> <size>100</size> <installNotifyURI>http://download.example.com/image.gif?id=image</installNotifyURI> </media> |
The file extension of the Download Descriptor should be set to .dd, and the MIME-Type must be set to application/vnd.oma.dd+xml.
Once a device downloads the descriptor, it will make some checks to see if the media object can be downloaded and installed/rendered, such as checking if the requierd free space exists and if it supports the specified file type, and then display the relevant information from the descriptor on the device’s browser (name, description, size, etc.). The user can then select to download the media object or cancel the download. Depending on what happens with the download, the installNotifyURI will be invoked, and the device will post to it the correspondant Status Code.
This is the list of possible Status Codes:
| Status Code | Status Message | Informative description of Status Code usage |
|---|---|---|
900 |
Success | Indicates to the service that the media object was downloaded and installed successfully. |
901 |
Insufficient memory | Indicates to the service that the device could not accept the media object as it did not have enough storage space on the device. This event may occur before or after the retrieval of the media object. |
902 |
User Cancelled | Indicates that the user does not want to go through with the download operation. The event may occur after the analysis of the Download Descriptor, or instead of the sending of the Installation Notification (i.e. the user cancelled the download while the retrieval/installation of the media object was in process). |
903 |
Loss of Service | Indicates that the client device lost network service while retrieving the Media Object. |
905 |
Attribute mismatch | Indicates that the media object does not match the attributes defined in the Download Descriptor, and that the device therefore rejects the media object. |
906 |
Invalid descriptor | Indicates that the device could not interpret the Download Descriptor. This typically means a syntactic error. |
951 |
Invalid DDVersion | Indicates that the device was not compatible with the “major” version of the Download Descriptor, as indicated in the attribute Version (that is a parameter to the attribute Media). |
952 |
Device Aborted | Indicates that the device interrupted, or cancelled, the retrieval of the media object despite the fact that the content should have been executable on the device. This is thus a different case from “Insufficient Memory” and “Non-Acceptable content). |
953 |
Non-Acceptable Content | Indicates that after the retrieval of the media object, but before sending the installation notification, the Download Agent concluded that the device cannot use the media object. |
954 |
Loader Error | Indicates that the URL that was to be used for the retrieval of the Media Object did not provide access to the Media Object. This may represent for example errors of type server down, incorrect URL and service errors. |
So, besides providing the Download Descriptor for the content download (which you will probably generate on the fly for each download), you will need the script which will be invoked on the installNotifyURI to process the status code. The following sample code is the relevant part of a script which could be used to process the status code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<?php $status_code = substr(trim(file_get_contents("php://input")), 0, 3); switch($status_code) { case '900': // Success!! Do whatever you need when it was correctly downloaded break; default: // Download failed, do whatever you need to do in case of failure break; } ?> |
For a complete description of this technology and the Download Descriptor itself, I highly recommend reading the OMA Download OTA Specification documents.
Appendix A: Streaming for Apple iPhone
Apple iPhone uses HTTP byte-ranges for requesting audio and video files. First, the Safari Web Browser requests the content, and if it’s an audio or video file it opens it’s media player. The media player then requests the first 2 bytes of the content, to ensure that the Webserver supports byte-range requests. Then, if it supports them, the iPhone’s media player requests the rest of the content by byte-ranges and plays it.
Thomas Thomassen has done a great job on his PHP Resumable Download Server, providing working PHP code which supports byte-range downloads.
The following sample code is the complete version for the one from the first part of the article, but including byte-range support by using the rangeDownload() function when the $_SERVER['HTTP_RANGE'] header is present on the device’s HTTP request. The rangeDownload() function is an exact copy&paste from Thomas Thomassen’s code (only the relevant part).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 |
<?php if (is_file($file)) { header("Content-type: $mime_type"); if (isset($_SERVER['HTTP_RANGE'])) { // do it for any device that supports byte-ranges not only iPhone rangeDownload($file); } else { header("Content-Length: ".filesize($file)); readfile($file); } else { // some error... } function rangeDownload($file) { $fp = @fopen($file, 'rb'); $size = filesize($file); // File size $length = $size; // Content length $start = 0; // Start byte $end = $size - 1; // End byte // Now that we've gotten so far without errors we send the accept range header /* At the moment we only support single ranges. * Multiple ranges requires some more work to ensure it works correctly * and comply with the spesifications: http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.2 * * Multirange support annouces itself with: * header('Accept-Ranges: bytes'); * * Multirange content must be sent with multipart/byteranges mediatype, * (mediatype = mimetype) * as well as a boundry header to indicate the various chunks of data. */ header("Accept-Ranges: 0-$length"); // header('Accept-Ranges: bytes'); // multipart/byteranges // http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.2 if (isset($_SERVER['HTTP_RANGE'])) { $c_start = $start; $c_end = $end; // Extract the range string list(, $range) = explode('=', $_SERVER['HTTP_RANGE'], 2); // Make sure the client hasn't sent us a multibyte range if (strpos($range, ',') !== false) { // (?) Shoud this be issued here, or should the first // range be used? Or should the header be ignored and // we output the whole content? header('HTTP/1.1 416 Requested Range Not Satisfiable'); header("Content-Range: bytes $start-$end/$size"); // (?) Echo some info to the client? exit; } // If the range starts with an '-' we start from the beginning // If not, we forward the file pointer // And make sure to get the end byte if spesified if ($range0 == '-') { // The n-number of the last bytes is requested $c_start = $size - substr($range, 1); } else { $range = explode('-', $range); $c_start = $range[0]; $c_end = (isset($range[1]) && is_numeric($range[1])) ? $range[1] : $size; } /* Check the range and make sure it's treated according to the specs. * http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html */ // End bytes can not be larger than $end. $c_end = ($c_end > $end) ? $end : $c_end; // Validate the requested range and return an error if it's not correct. if ($c_start > $c_end || $c_start > $size - 1 || $c_end >= $size) { header('HTTP/1.1 416 Requested Range Not Satisfiable'); header("Content-Range: bytes $start-$end/$size"); // (?) Echo some info to the client? exit; } $start = $c_start; $end = $c_end; $length = $end - $start + 1; // Calculate new content length fseek($fp, $start); header('HTTP/1.1 206 Partial Content'); } // Notify the client the byte range we'll be outputting header("Content-Range: bytes $start-$end/$size"); header("Content-Length: $length"); // Start buffered download $buffer = 1024 * 8; while(!feof($fp) && ($p = ftell($fp)) <= $end) { if ($p + $buffer > $end) { // In case we're only outputtin a chunk, make sure we don't // read past the length $buffer = $end - $p + 1; } set_time_limit(0); // Reset time limit for big files echo fread($fp, $buffer); flush(); // Free up memory. Otherwise large files will trigger PHP's memory limit. } fclose($fp); } ?> |
Appendix B: General Content Descriptor (GCD)
In the mobile world, developers need not only to deal with devices peculiarities, but many times carriers and their networks have peculiarities too.
GCD is one of these cases. GCD is a technology specific to Sprint (US carrier). If you don’t provide content for Sprint users, there’s almost no chance that you will need to use it.
But it’s good to know about it, I’ve also seen Iusacell from Mexico sell some Sprint phones, which need to make use of GCD.
A GCD is very similar to a JAD file. It’s a simple text file that contains information which allows the download of content to Sprint PCS Vision-capable devices.
For downloading content to these devices you MUST use a GCD.
This is the content of a sample GCD file:
|
1 2 3 4 5 6 |
Content-Type: audio/midi Content-Name: My Ringtone Content-Version: 1.0 Content-Vendor: Company Name Content-URL: http ://wap.mydomain.tld/download.php/123456abcdef/myringtone.mid Content-Size: 3257 |
Each part of the file is very important for the phone in order to be able to download the content.
| Name | Definition |
|---|---|
Content-Type |
The content’s MIME-Type |
Content-Name |
The name of the file, which will show up on the phone |
Content-URL |
The URL where to download the content from |
Content-Size |
The content size in bytes. Must be the exact size of the content file, or the phone will display an error when downloading it |
It is very important that the GCD is created correctly, respecting the needed line-breaks. If not, it will not work on many devices.
Here is sample code for creating a GCD:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<?php $gcd_file = $path_to_gcd_file.'/download.gcd'; $handle = fopen($gcd_file, 'a'); fwrite($handle, "nContent-Type: $mime_typen"); fwrite($handle, "Content-Name: $content_namen"); fwrite($handle, "Content-Version: 1.0n"); fwrite($handle, "Content-Vendor: Company Namen"); fwrite($handle, "Content-URL: $download_urln"); fwrite($handle, "Content-Size: $file_sizen"); fclose($handle); ?> |
Conclusion
Serving content for mobile devices is a whole different world compared to the desktop Web.
In this article I’ve presented some possible ways on how to deliver content successfully to mobile devices. Depending on your environment and/or needs you may want to serve content directly as filesystem files or by streaming them – this is something you must analyse, and determine what is the best option for your specific application. You may even have additional needs which are not described here, so you must think what are your needs and requirements and act according to them.
You may also want to use OMA Download or not, but I would recommend to use it. Despite the fact that it may not be mandatory on your scenario, it can be very helpful for support purposes and also for statistical reasons.
So take your time, think what’s the best option for you, and happy content delivering!
18 Comments
Thanks! Will do.
Toni
Hi Juan
I’m pretty new to php however, I’ve grasped the logic to the I-Phone code. My only question would be what do I send as a parameter for the $_SERVER[‘HTTP_RANGE’] header? What parameter/s would you suggest for initial testing purposes?
Also, I’ve come across some information that says in order to “stream” or progressive download 3pg or any video file from a php script, one would have to rearrange the MOOV ATOMs to the beginning of the file. Is there validity to this? Your feedback is most appreciated!
Regards,
Toni
Hi Toni, sorry for the delay on replying.
Basically what you have to do is start sending the file content by chunks, follow the code and you should be able to reproduce it, it’s quite commented also explaining what it does.
Regarding 3GP files I’m sorry but really don’t know. so far I’ve only tested it with MP3 files.
Juan Nin – Manager of Software Development
3Cinteractive – Mobilizing Great Brands
http://www.3cinteractive.com
Juan:
We’re trying to get this php workaround to work, but are having problems. Are you available to help? Please email me -> chessdev[&}gm@il 🙂 Paid help of course!
Thanks,
Erik
Chess[.]com
Glad to see my PHP Download Server is of use for other people.
If anyone implement multi byte-range and would like to share back then drop me a message and I can add it to the download on my site.
-Thomas Thomassen
I’ve just been trying to get content streaming to an iPad / iPhone via a PHP script and came across this script which seemed to fit my requirements nicely.
However, there’s a typo on line 67 which I spent 3 hours looking for – so in case you’ve come here looking for a way to stream to iDevices then instead of using:
[code]if ($range0 == ‘-‘) {[/code]
use:
[code]if ($range == ‘-‘) {[/code]
It may save you some heartache / hairpulling 😉
Hello,
I try to integrate this code inside my web page to play video (mp4 files) on an ipad but I don’t know how to use it.
Somebody can help me ?
I hosted my web server on my Synology NAS and I work with wordpress.
The Range Download function seems to die if more than one client is accessing it simultaneously or if one browser loads the same video in two spots. Perhaps this is partly the browsers fault as well.
Basically if I setup two html5 video tags on the same page referencing the same video via a php page that ends up using range download I get very weird results. Only one will show up at first. It will play fine. If I try and jump to a new time it will stall and the second video will start playing from a random position. I’m having a lot of trouble trying to debug this. Any advice?
It appears to work fine if they both use the normal download where it just calls read file.