Entity view (Content)

Uploading videos with Brightcove FTP Batch Provisioning

By kmoll
Dec. 3, 2014
Brightcove is a leading platform for cloud-based online video hosting and publishing. They have grown in popularity over the years because, for sites that deal with a lot of media, storing and serving those videos takes up too many resources. You need large amounts of storage to keep every video and a large amount of bandwidth if you are going to serve the videos to millions of users over the web. Many Drupal sites take advantage of the Brightcove platform and there is a contributed Brightcove module. It integrates their API and creates a custom field that will upload files to Brightcove through their Media API. This is a great module that works really well. The problem that we found is that for large files, this upload process can take a long time, and during the process you are stuck in that workflow until you get a response back from Brightcove. For sites that don’t upload a lot of videos or only use short clips, this may not be a problem. But for large media sites using large hi res files, this could slow down an editors workflow. While working on a project to move Time Inc. over to Drupal, decreasing time spent within an editor’s typical workflow was the main goal. Time spent sitting around waiting for things to happen in the background was time wasted. When we started working on integrating their platform with Drupal we wanted to make sure that editors can move on to other tasks rather than waiting for videos to upload, weather it takes 1 minute or 5. So we had to have a solution that would allow the editors to upload the video locally, then have the videos uploaded to Brightcove in a separate process. There are many different ways we could have accomplished this, but Time Inc. already had a system in place to handle communicating with Brightcove. Their old CMS didn’t have native integration with Brightcove, so they put in place a system that took videos uploaded to the CMS and uploaded videos in the background using Brightcove’s FTP batch provisioning. (Note: FTP batch provisioning is only available for Video Cloud Pro and Enterprise customers). You can read an in depth description of FTP batch processing here but essentially it allows you to upload or edit multiple videos at a time through FTP. You need to send any assets (video files and image files) that need to be upload along with a manifest file (written in xml) to describe the meta data associate with the file. The manifest defines which asset the video is, what asset the thumbnail and video still are, a title and description of the video etc, and should include a unique reference ID. If the ID already exists in Brightcove, then the video will be updated with the metadata in the manifest file. While we were discussing the different ways we can take the upload process out of the editors workflows we decided to utilize the system they already had in place. Then all we had to do was to feed the system the files it needs. All it needed was an XML manifest file and the assets associated. This was really easy to implement in Drupal through the entity creation/editing process. We created a Video content type and give it fields for every piece of metadata we needed to capture and send to Brightcove, and we gave it a few file upload fields. One field for the video itself, one for the image thumbnail and one for the video still. If these files aren’t specified, Brightcove will process the video and create a video still and thumbnail out of the first frame of the video. To create the manifest file, we attach a submit function to the node form and loop through the values to create the XML file using PHP’s DOMDocument class. When we submit the node form, we create this manifest file and move all the assets for the video to a specific location on the server. The existing system gets notified when the contents of that folder change and it simply grabs the manifest and files, and transfers it to Brightcove. From there Brightcove will upload, import, then render the video. This whole process can take a few minutes. We wanted to make sure we updated the video node’s status so that an editor will always be aware of where in the process the video is. So we used the Services module to create a service to update the nodes. Brightcove’s FTP Batch Provisioning system sends notifications to a callback url that you specify within the manifest file. Our internal system also sent notifications through this service such as setting the status to “Uploaded” to tell the editor that the system has picked up the video and passed it to Brightcove. It will also set the status to “Failed Upload” if it wasn’t able to upload the assets to Brightcove. Since our internal system had to talk to Drupal, we decided that we would have Brightcove talk to the internal system so we had one point of contact with Drupal rather than having both systems have the ability to update the node. This also allowed us to keep the Drupal instance behind a firewall and only expose a subset of the internal system to the Internet. Once we get the final word that the video is rendered which means Brightcove is able to serve it, we get a notification from Brightcove and the internal system passes that along to Drupal. It updates the status to “Complete” and published the video. Now we can imbed or attach the video into our articles and content. By utilizing Brightcove’s Batch FTP Provisioning platform, we were able to greatly reduce the time an editor sat around waiting for videos to upload. Now, while a video is uploading, they can move on and create or edit other pieces of content. We also tied the service into the notification system so editors didn’t have to check back in to see if the video upload was successful. When the video was uploaded and rendered, editors were sent a notification that the video was published and ready to push to the site. As mentioned earlier, there are other ways to remove the video uploading from the editorial workflow. If you run a smaller site and don’t have the sophisticated technical environment of a company like Time Inc. you could still implement a similar approach with using only Drupal by leveraging cron and tools like ssh2_sftp to replicate a similar process.
Post Tags: