Media Extensions

Important:

This page describes an API that has been deprecated as of webOS 1.4 and will eventually be removed. For information on the current API, refer to the Audio and Video doc pages.

Overview

To support non-standard features, webOS has added extensions to HTML5 media objects, including mnemonics for several structures and values defined in HTML5, plus a .palm sub-object to the Audio and Video objects for webOS-specific features.

Media Events

These events are generated by HTMLMediaElement instances.

Applications must explicitly register event handlers to receive them, which is standard with HTML events. At any time after creating an Audio or Video object, the normal addEventListener(event, callback) call is used to register for an event.

Note:

The third parameter to addEventListener is currently ignored for HTMLMediaElement objects.

Javascript define Description
Media.Event.ABORT Corresponds to HTMLMediaElement event of the same name.
Media.Event.CANPLAY Corresponds to HTMLMediaElement event of the same name.
Media.Event.CANPLAYTHROUGH Corresponds to HTMLMediaElement event of the same name.
Media.Event.CANSHOWFIRSTFRAME Corresponds to HTMLMediaElement event of the same name.
Media.Event.DURATIONCHANGE Corresponds to HTMLMediaElement event of the same name.
Media.Event.EMPTIED Corresponds to HTMLMediaElement event of the same name.
Media.Event.ENDED Corresponds to HTMLMediaElement event of the same name.
Media.Event.ERROR Corresponds to HTMLMediaElement event of the same name.
Media.Event.LOAD Corresponds to HTMLMediaElement event of the same name.
Media.Event.LOADEDFIRSTFRAME Corresponds to HTMLMediaElement event of the same name.
Media.Event.LOADEDMETADATA Corresponds to HTMLMediaElement event of the same name.
Media.Event.LOADSTART Corresponds to HTMLMediaElement event of the same name.
Media.Event.PAUSE Corresponds to HTMLMediaElement event of the same name.
Media.Event.PLAY Corresponds to HTMLMediaElement event of the same name.
Media.Event.PROGRESS Corresponds to HTMLMediaElement event of the same name.
Media.Event.SEEKED Corresponds to HTMLMediaElement event of the same name.
Media.Event.SEEKING Corresponds to HTMLMediaElement event of the same name.
Media.Event.STALLED Corresponds to HTMLMediaElement event of the same name.
Media.Event.TIMEUPDATE Corresponds to HTMLMediaElement event of the same name.
Media.Event.WAITING Corresponds to HTMLMediaElement event of the same name.
Media.Event.X_PALM_CONNECT The JavaScript library has connected to the media process and is ready for commands.
Media.Event.X_PALM_DISCONNECT The JavaScript library has disconnected from the media process (probably due to error).
Media.Event.X_PALM_RENDER_MODE Notification that the render mode (video sink) has changed.
Media.Event.X_PALM_SUCCESS Low-level notification that the message sent to the media process succeeded (not normally used by applications).
Media.Event.X_PALM_WATCHDOG No response was received from the media process for a prolonged time (error condition).

Palm-Specific Events

The preceding events with the "X_PALM_" prefix are Palm extensions to the normal HTMLMediaElement event list. The connect, disconnect, and watchdog events are important for correct operation and error handling.

This section discusses the following events:

Connect Event

The connect event is sent after establishing the connection to the mediaserver. Applications must not try to play media or set the .src of an audio object until after receiving this event because if the object is not associated with the mediaserver before attempting this the commands may be lost. It is acceptable to call addEventListener() before receiving the connect, but applications cannot do anything else.

Example of connect callback:

function loadIt() {
  aGlobal = new Audio();
  aGlobal.addEventListener('x-palm-connect',
      function(evt){
          evt.target.palm.audioClass = "media";
          evt.target.addEventListener(...);
          evt.target.addEventListener(...);
          evt.target.addEventListener(...);
          evt.target.src="/foo.mp3";
      },
      false);
}  

Disconnect/Watchdog Event

Applications should listen for X_PALM_DISCONNECT and X_PALM_WATCHDOG events because they signify a fatal error communicating with the media server. If either event is received, an application should stop using the current HTMLMediaElement, remove it from the DOM, and attempt to create a new object.

Error Codes and Additional Information

The following errors are generated by Audio, Camera, and Video objects.

Definition Description
MediaError.MEDIA_ERR_ABORTED Equals HTML5 aborted error value.
MediaError.MEDIA_ERR_CAPTURE Returned from Camera object if capture failed.
MediaError.MEDIA_ERR_DECODE Equals HTML5 decode error value.
MediaError.MEDIA_ERR_NETWORK Equals HTML5 network error value.
MediaError.MEDIA_ERR_SAVE Returned from Camera object if save failed.

An application must register for error events (addEventHandler(Media.Event.ERROR, ...)) to get error callbacks. The event passed to the callback (through the first argument) contains the type of the event, which in this case is "error", and the target for the event.

Extended Error Information

Additional error information, beyond that defined by the HTML5 specification, is also passed to the application through the .palm.errorDetails object.

There are three fields in errorDetails: errorClass, errorCode, and errorValue. These fields contain the raw error information from the media server, as defined in the following table. An application must first check the standard HTML5 error value to determine whether the extended information is relevant.

All errors have the 3 MS Bytes set to 0xE00505.

ErrorClass ErrorCodes Notes
kMediaDecodeError (0xE0000501) kMediaDecodeErrorFileNotFound (1), kMediaDecodeErrorBadParam(2), kMediaDecodeErrorPipeline (3), kMediaDecodeErrorUnsupported (4), kMediaDecodeErrorNoMemory (5) All errors related to the parameters or the stream.
kMediaNetworkError (0xE0000502) kMediaNetworkErrorHttp (6), kMediaNetworkErrorRtsp (7), kMediaNetworkErrorMobi (8), kMediaNetworkErrorOther(9),
kMediaNetworkErrorPowerDown(12)
All errors related to the network or streaming (except 12 which is generic).
kMediaCaptureError (0xE0000503) kMediaCaptureErrorCaptureFailed (10), kMediaCaptureErrorImageSaveFailed (11) All capture related errors.

Media Audio Class Tagging

The audio class of a piece of media defines how the system deals with the audio for the content. For example, the audio route sending the sound (headset, back speaker, front speaker) or how the sound is treated when receiving a notification or ringtone (e.g., mute, pause, reduce volume).

Both Audio and Video objects include a .palm.audioClass parameter. You should set this parameter before starting playback of media.

Example:

var obj = new Audio();
obj.palm.audioClass = Media.AudioClass.MEDIA;
obj.src="/file.mp3"; 

Special Considerations for Audio Tagged as MEDIA

Most audio classes simply control the volume, speaker the mixing. However, streams tagged Media.AudioClass.MEDIA have some extra behavior, as follows:

Applications using the Media.AudioClass.MEDIA audio class should anticipate their stream being paused unexpectedly. Therefore, it is important that their UI updates appropriately, when playback pauses/resumes. An application should register for the Media.Event.PAUSE and Media.Event.PLAY events to correctly update their UI to reflect the playback state.

Additional Streaming Data

The section discusses the properties on the Palm extension to the HTMLMediaElement. They provide additional information about the buffering and playback rate of streaming content.

Bitrate

The .palm.bitrate object on an HTMLMediaElement has the following properties:

These properties are set with data from the stream being played. The average and maximum bitrates are set from data in the container for the stream. After receiving the CANPLAY event, the values are considered valid. If there is no bitrate information present in the container (they are optional), both audio.estimate and video.estimate are set to true.

Example:

var a = new Audio();
...
// An audio stream shouldn't have video
if (a.palm.bitrate.audio.estimate == false) {
  Mojo.log(
      "bitrates: ",
      a.palm.bitrate.audio.average,
      "/",
      a.palm.bitrate.audio.maximum);
} 

Buffering Rate

The HTTP source element provides an average of the download rate from the server; palm.bufferingRate holds the buffering bytes/second value for the current stream.

Consumption Rate

The HTTP source element provides the average data rate being pulled by the downstream component; palm.consumptionRate holds the average pull rate.

Setting Properties on the GStreamer Pipeline

This is a low-level API that allows properties to be set on an open mediaserver stream.

To passdown properties to mediaserver components, use palm.setStreamProperty(property, value). Currently the following are exposed:

Releasing the Video Sink

When an application sets the "video sink", it controls the location of the video for the playing content. When an application is not in the foreground, it loses control of the video sink, so another foreground application can use it to display video.

Applications can release the video sink themselves, but are not required to do so.

Command Description
obj.palm.freeVideoSink() Releases the video sink. The application can continue to play video, but the UI does not updated.
obj.palm.renderMode Controls the destination where the video sink outputs the video. There are three valid settings:
  • video.palm.renderMode = 'overlay' - Renders normally to the video sink.
  • video.palm.renderMode = 'rgb' - Grabs a frame of video, but continues to hold the video sink open.
  • video.palm.renderMode = 'null' - Releases the video sink. The application can continue to play video, but the UI does not updated.

An application does not need to do anything to release the video sink, this happens automatically when the application loses focus (gets carded). However, the application developer may want to pause the video playback when carded because video is not visible.

Video Scaling in Full Screen Mode

Note:

Video Scaling is only implemented for fullscreen video.

Video fits into the display rectangle according to the value set in fitMode. If fitMode is set to "fit", the video only stretches until the larger dimension fills the window. There may be black bars in the other dimension.

If fitMode is set to "fill", the video size scales until the entire screen contains the video, though this may result in some cropping on one dimension.

var video = new Video("file.mp4");
...
function someButtonHandler(toggleState){
  // Assuming toggleState is a boolean attached to the state
  // of an onscreen button...
  if (toggleState){
      // When true we want to fill the screen
      video.palm.fitMode = 'fill';
  }  else {
      video.palm.fitMode = 'fit';
  }
} 

Using a Non-Fullscreen View Rectangle

To use a non-fullscreen view rectagle, use the following code:

video.palm.windowed = true;
video.palm.setViewRectangle(left, top, width, height);

Setting the Video Orientation

The orientation of the video is controlled independently of the window or devices orientation. By default, video plays with the assumption that the device rotates left (counter-clockwise) degrees. Set the video.palm.windowOrientation to ensure the correct orientation for the carded video. For example, video.palm.windowOrientation = 'left'. Valid values are "up", "down", "left", and "right".

Adding a Media Object to the DOM

Using the standard HTML5 constructor (for example, var a = new Audio();), you create a media object and append it to the DOM of the current document. That API does not give you much control over where in the DOM the document is added and may cause problems with applications containing multiple scenes.

The preferred way to create a media object is to extend an existing HTML div object by using AudioTag.extendElement(), as shown in the following code:

<html>
  <head>
  <script type="text/javascript">
  function myExampleFunction(){
      // Do some setup
      var audioElem = AudioTag.extendElement($('someDiv'));
      // Do standard HTML5 media actions with the object
      audioElem.src="http://domain.dom/path/to/audio.mp3";
      ...
  }

  // This function is called in response to some event that requires playback audio
  function myFooHandler(){
      // Get reference to the audio object by referring to HTML DIV
      var audio = $('someDiv');
      audio.play();
  }
  </script>
  </head>
  
  <body>
  <div id="someDiv" controls="false" autoplay="false"></div>
  </body>
</html> 

When you do this, you add the media functionality to an existing <div> in the page/scene. This means that after popping the scene and removing the scene DOM, the audio object is also cleaned up.

Note:

The audio stops playing after removing the div from the DOM.

Constructors for Audio and Video objects

The expected way to create an Audio or Video object is to use the corresponding AudioTag.extendElement() and VideoTag.extendElement() methods. However, a default constructor for an audio object does exist (new Audio()). Calling new Video() generates an exception because the Video object requires that a <div> tag be specified representing the size and position of the video on the screen.

The constructor that supports passing the div (as a name or element) exists for both Audio and Video classes, but is deprecated. new Video(url, div) and new Audio(url, div) are currently both defined.