Working with PDF RichMedia Annotations
By Apryse | 2013 Sep 11
7 min
Tags
tutorial
annotation
video/audio
RichMedia annotations can be used to embed videos and sound files in PDF documents. When opened in a compliant reader, the user can play them back directly as part of the page content, usually by just clicking on the video. This blog will examine how a RichMedia annotation is embedded in PDF, how PDFNet can be used to extract and create the movie data for playback.
How to Open and Playback a PDF RichMedia Annotation
The simplest way to play back a movie annotation is to take advantage of the host operating system’s APIs that are capable of playing video content. For example, both iOS and Android have built in controls that take video files and present them to the user with controls to start and stop playback. The input files for these controls can be created by extracting the raw video data from the RichMedia annotation and saving it (temporarily) to disk. The basic steps are:
- Extract the movie file from the RichMedia annotation and save it to disk. (You may want to check that the operating system supports the particular video’s encoding.)
- Add a video playback widget directly over the annotation.
- Begin playback.
Extracting the Media File
Here is a closeup look at what is actually inside the PDF file. This low-level content is abstracted by PDFNet, but it may be of interest to know what’s going on under the hood. The media file information can be found in the “Assets” name tree, which is inside the RichMediaPresentation dictionary of the RichMedia annotation entry. This raw PDF content shows what’s in the Assets entry:
29 0 obj % Assets name tree
<< /Names [ (video.mp4) 30 0 R (Flash.swf) 31 0 R ] >>
endobj
30 0 obj % File specification dictionary for video file
<< /Type /Filespec
/F (video.mp4)
/UF (video.mp4)
/EF >> % Stream containing the video file
>>
endobj
31 0 obj % File specification dictionary for SWF file
<< /Type /Filespec
/F (Flash.swf)
/UF (Flash.swf)
/EF << /F 41 0 R >> % Stream containing the Flash file
>>
endobj
40 0 obj % Embedded file stream for video file
<< /Type /EmbeddedFile % video.mp4 /Length ... /Filter ... >>
stream
- DATA for video.mp4 -
endstream
endobj
To extract the video.mp4 file we need to iterate through the annotation assets tree until we find the video content. The following code performs this (code is in Java, but any of PDFNet’s supported languages could be used as well):
Note: the code snippets that follow below assume that a pdftron.PDF.PDFViewCtrl view is being used to display PDF documents, represented as the “mPDFView” member variable.
Annot mRichMediaAnnot;
public File getRichMediaFile() {
File mediaFile = null;
try {
Page page = mPDFView.getDoc().getPage(mPDFView.getCurrentPage());
int numAnnots = page.getNumAnnots();
for (int i = 0; i < numAnnots; ++i) {
mRichMediaAnnot = page.getAnnot(i);
if (!mRichMediaAnnot.isValid()) {
continue;
}
// Check if the current annotation is a RichMedia type
if (mRichMediaAnnot.getType() == Annot.e_RichMedia) {
Obj annotObj = mRichMediaAnnot.getSDFObj();
Obj rmcObj = annotObj.findObj("RichMediaContent");
if (rmcObj != null) {
NameTree assets = new NameTree(rmcObj.findObj("Assets"));
if (assets.isValid()) {
NameTreeIterator j = assets.getIterator();
for (; j.hasNext(); j.next()) {
// We iterate through the assets tree and get the name
// of the files
String asset_name = j.key().getAsPDFText();
// At this point we can perform a pre-check if the file
// is supported by looking at its extension, and jump
// to the next one if it is not the case
if (isMediaFileSupported(asset_name)) {
mediaFile = new File(getApplicationContext().getExternalFilesDir(null), asset_name);
FileSpec file_spec = new FileSpec(j.value());
Filter stm = file_spec.getFileData();
if (stm != null) {
stm.writeToFile(mediaFile.getAbsolutePath(), false);
break;
}
}
}
}
}
break;
}
}
} catch (PDFNetException e) {
e.printStackTrace();
}
return mediaFile;
}
// Check if the video file is supported on the current platform.
public boolean isMediaFileSupported(String fileName) {
String[] SUPPORTED_FORMATS = {".3gp", ".mp4", ".ts", ".webm", ".mkv"};
int idx = fileName.lastIndexOf(".");
return ((idx != -1) && Arrays.asList(SUPPORTED_FORMATS).contains(fileName.substring(idx)));
}
The method getRichMediaFile() iterates through all the annotations of the page being viewed, checking for the first one that has the RichMedia type. If the annotation is found, it tries to extract the media to a (temporary) file. The isMediaFileSupported() method is used to do a pre-check on the file and see if it is supported on the current platform (for example, the supported formats on Android can be seen here).
Playing Back the Media File
Once you extract the file it can then be passed to appropriate APIs that are available on your platform. For example, you can open the file using an external media player, or in case you are creating an Android app, you can use a VideoView widget, etc.
The code below shows how to playback the video file using a VideoView widget on Android. The VideoView is sized and positioned according to the annotation on the page (another option, for example, would be to launch an external media player):
VideoView mVideoView;
private void setupAndPlayMedia(File mediaFile, Annot richMediaAnnot) {
mVideoView = new VideoView(mPDFView.getContext());
mVideoView.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
@Override
public void onPrepared(MediaPlayer mp) {
new StartVideoViewTask().execute();
}
});
mVideoView.setOnCompletionListener(new MediaPlayer.OnCompletionListener() {
@Override
public void onCompletion(MediaPlayer mp) {
Toast.makeText(mPDFView.getContext(), "End of video playback", Toast.LENGTH_SHORT).show();
}
});
// Adjust size and position of the VideoView widget
adjustViewPosition(mVideoView, richMediaAnnot);
mVideoView.setVideoPath(mediaFile.getAbsolutePath());
mVideoView.setMediaController(new MediaController(mPDFView.getContext()));
// Add the view to PDFViewCtrl
mPDFView.addView(mVideoView);
}
// This method gets the size/position information of the RichMedia annotation
// and sets the layout of the view with the same size.
private void adjustViewPosition(View view, Annot annot) {
if (view != null && annot != null) {
try {
pdftron.PDF.Rect r = annot.getRect();;
double x1 = r.getX1();
double y1 = r.getY2();
double x2 = r.getX2();
double y2 = r.getY1();
// The annotation size is expressed on page space, so we first need to
// convert the positions to screen space to set the view layout.
double pts1[] = mPDFView.convPagePtToScreenPt(x1, y1, mPDFView.getCurrentPage());
double pts2[] = mPDFView.convPagePtToScreenPt(x2, y2, mPDFView.getCurrentPage());
x1 = pts1[0];
y1 = pts1[1];
x2 = pts2[0];
y2 = pts2[1];
int sx = mPDFView.getScrollX();
int sy = mPDFView.getScrollY();
int anchor_x = (int) (x1 + sx + 0.5);
int anchor_y = (int) (y1 + sy + 0.5);
view.layout(anchor_x, anchor_y, (int) (anchor_x + x2 - x1 + 0.5), (int) (anchor_y + y2 - y1 + 0.5));
} catch (PDFNetException e) {
}
}
}
// This async task is used to start the video playback
private class StartVideoViewTask extends AsyncTask<Void, Void, Void> {
@Override
protected void onProgressUpdate(Void... values) {
if (mVideoView != null) {
mVideoView.start();
}
}
}
The setupAndPlayMedia() creates a VideoView widget object and its size and position are calculated based on the RichMedia annotation present in the page using the adjustViewPosition() method. For example, you can use the following code to start and stop playing back a RichMedia annotation:
// Start playback
File mediaFile = getRichMediaFile();
if (mediaFile != null) {
setupAndPlayMedia(mediaFile, mRichMediaAnnot);
}
// Stop playback
if (mVideoView != null) {
mVideoView.stopPlayback();
mPDFView.removeView(mVideoView);
mVideoView = null;
}
Creating a RichMedia Annotation
The code snippet below shows how to create a RichMedia annotation using the PDFNet API. It will create a new document with a RichMedia annotation placed in the lower left corner of the first page. The annotation object is first created, and then filled with all the information required by the PDF specification.
PDFDoc doc = new PDFDoc();
Page page = doc.pageCreate();
// Create a new RichMedia annotation and populate it with the
// required information
Annot annot = Annot.create(doc, Annot.e_RichMedia,
new Rect(50, 50, 200, 200));
Obj annotObj = annot.getSDFObj();
Obj rmcObj = annotObj.putDict("RichMediaContent");
// Embed the video file
String video_file = "video.mp4";
FileSpec fs = FileSpec.create(doc, "D:/" + video_file);
fs.getSDFObj().putString("F", video_file);
fs.getSDFObj().putText("UF", video_file);
// Create/find the asset NameTree
NameTree assets = NameTree.create(doc, "Assets");
assets.put(video_file.getBytes(), fs.getSDFObj());
rmcObj.put("Assets", assets.getSDFObj());
Obj config = doc.createIndirectDict();
config.putName("SubType", "Video");
Obj instance = doc.createIndirectDict();
instance.put("Asset", fs.getSDFObj());
Obj instances = doc.createIndirectArray();
instances.pushBack(instance);
config.put("Instances", instances);
Obj configs = doc.createIndirectArray();
configs.pushBack(config);
rmcObj.put("Configurations", configs);
// Let's create an appearance for the annotation
// using an image
ElementBuilder builder = new ElementBuilder();
ElementWriter writer = new ElementWriter();
writer.begin(doc);
Image image = Image.create(doc, "D:/image.jpg");
Element element = builder.createImage(image,
Matrix2D.identityMatrix());
Rect bbox = element.getBBox();
writer.writeElement(element);
Obj appearance = writer.end();
appearance.putRect("BBox", bbox.getX1(),
bbox.getY1(), bbox.getX2(), bbox.getY2());
appearance.putName("Subtype", "Form");
annot.setAppearance(appearance);
// Add the annotation to the document
page.annotPushBack(annot);
doc.pagePushBack(page);
doc.save("document.pdf", SDFDoc.e_linearized, null);
doc.close();
Further Reading
More information on RichMedia annotations can also be found in our forums, including the following two discussions:
Tags
tutorial
annotation
video/audio
Apryse
Related Products
Share this post
