From 20bf031db719f7baa4c6e74734fc999061e08fe2 Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Thu, 19 Jul 2012 21:15:10 +0200
Subject: Bug 599 - FBObject / Offscreen Support - Part 1
- New FBObject implementation handling FBO and it's attachments *** API CHANGE: Util -> Core ***
while it's size and sample-count can be reconfigured on the fly.
- com.jogamp.opengl.util.FBObject -> com.jogamp.opengl.FBObject
- agnostic to texture unit
- separate attachments using OO hierarchy reflecting FBO
- handling MSAA and blitting
- no FBO destruction for reconfig (attach/detach)
- New GLFBODrawableImpl impl. an FBObject based GLDrawable
- Instantiated by a dummy native surface (onscreen and invisible)
hooked up to a dummy GLDrawable, which is the delegation for context creation.
- Utilizies ProxySurface.UpstreamSurfaceHook for dummy surface
avoiding specialization for native platforms.
- TODO: Allow to utilize common surface interface as a
dummy-surface to supporting API seperation of
windowing/GL. The latter allows impl. of createGLDrawable(NativeSurface)
with FBO.
- New OffscreenAutoDrawable (extends GLAutoDrawableDelegate)
for all offscreen drawables. Shall replace GLPbuffer.
- New GLCapabilities*.isFBO() / setFBO(boolean) to request FBO offscreen,
similar to isPBuffer(). Rule: if both are requested, FBO shall be favored.
- GLContext adds raw FBO availability query (min. FBO avail),
FBObject contains fine grained queries (TODO: Move parts to GLContext for efficiency).
- Add framebuffer tracking, allowing fast querying:
- GLBase/GLContext:
public int getBoundFramebuffer(int target);
public int getDefaultDrawFramebuffer();
public int getDefaultReadFramebuffer();
- GLContextImpl
public final void setBoundFramebuffer(int target, int framebufferName)
.. called by GL impl bind framebuffer
- GL: getDefaultDrawFramebuffer(), getDefaultReadFramebuffer()
Adding default framebuffer queries being issued by
GL.glBindFramebuffer(target, 0) w/ a default framebuffer, o.e. zero.
This allows a transparent use of a custom FBO even in case the applications
attempts to reset FBO to zero.
Value flow: GL <- GLContext <- GLDrawable,
- GLCapabilities handle fbo/pbuffer seperate, don't disable the other
- GLContext/GL track read/write framebuffer to be queried by FBObject
to determine whether to bind/unbind a framebuffer
- Test cases for multiple FBO w/ and w/o MSAA
Other Features:
- New interface ProxySurface.UpstreamSurfaceHook,
allowing to hook an upstream surface of unknown type
providing lifecycle and information (size, ..) callbacks.
Used for all new dummy NativeSurface impl and SWT GLCanvas.
- GLContext -> GLDrawable propagation context/drawable lifecycle
via ProxySurface.UpstreamSurfaceHook allowing dynamic resources
to react (create, init, ..)
- contextRealized()
- contextMadeCurrent()
- SurfaceChangeable -> MutableSurface
currently only contains setting the surface handle.
TODO: May need to move ProxySurface.UpstreamSurfaceHook -> MutableSurface.UpstreamSurfaceHook,
allowing other impl. classes (NEWT OffscreenWindow) to utilize the new
upstream hookup mechanism - will allow FBO/Dummy window to work.
- SWT GLCanvas using ProxySurface.UpstreamSurfaceHook for proper size
propagation.
- New GLAutoDrawable::getUpstreamWidget(), allowing GLEventListener
to fetch the owning Java side UI element (NEWT, SWT, AWT, ..).
- GLDrawableFactory: Removed createOffscreenSurface() - unused and not GL related
- EGLDrawableFactory handles device/profile avail. mapping
while actually creating context/drawable.
This allows us to learn whether the ES context is software/hardware as well as FBO avail.
- EGLDrawable: Removed secret buckets of EGL configs :)
Employ native surface (X11, WGL, ..) to EGL 'mapping' in
EGLDrawableFactory utilizing new EGLUpstreamSurfaceHook (implements ProxySurface.UpstreamSurfaceHook).
Other Bugs:
- Add CTX_OPTION_DEBUG to ctx/extension cache key since only a debug ctx
may expose the ARB debug capability.
This bug caused lack of ARB/AMD debug functionality.
- Fix GLProfile deadlock (debug mode, w/ EGL/ES, no X11),
dump availability information _after_ lock.
- ImmModeSink draw(): Use GL's glDrawElements(..), don't cast for GL2ES1.
Fixes use for GL2ES2.
- Fix KeyEvent.getKeyChar() comment (-> only stable for keyTyped(..))
Misc:
- Refined alot of API doc
- New GLExtensions holds commonly used GL extension strings,
allows better referencing and usage lookup.
- Move GL (interface) decl. to GLBase
- GLBuffers: Cleanup API doc (format, types)
- TextureIO: Add PAM and PPM static suffix identifier
- GLCapabilities getNumSamples() returns 0 if sampleBuffers is disabled, this seems to be more natural.
- finalized a lot
---
src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java | 1 -
1 file changed, 1 deletion(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 5ee2104a3..6b41c0bc8 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -102,7 +102,6 @@ import javax.media.opengl.GL;
*/
public interface TextureSequence {
public static final String GL_OES_EGL_image_external_Required_Prelude = "#extension GL_OES_EGL_image_external : enable\n";
- public static final String GL_OES_EGL_image_external = "GL_OES_EGL_image_external";
public static final String samplerExternalOES = "samplerExternalOES";
public static final String sampler2D = "sampler2D";
--
cgit v1.2.3
From d5599155b28f63a83112d4a4268c2cca246c9e28 Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Mon, 15 Jul 2013 13:39:44 +0200
Subject: StringBuffer -> StringBuilder
---
src/jogl/classes/com/jogamp/opengl/util/GLPixelBuffer.java | 4 ++--
src/jogl/classes/com/jogamp/opengl/util/awt/AWTGLPixelBuffer.java | 4 ++--
src/jogl/classes/com/jogamp/opengl/util/glsl/ShaderUtil.java | 4 ++--
src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java | 4 ++--
src/jogl/classes/jogamp/opengl/windows/wgl/WGLGLCapabilities.java | 4 ++--
src/nativewindow/classes/javax/media/nativewindow/Capabilities.java | 2 +-
.../classes/javax/media/nativewindow/CapabilitiesImmutable.java | 2 +-
src/newt/classes/com/jogamp/newt/MonitorMode.java | 4 ++--
src/newt/classes/jogamp/newt/driver/android/NewtVersionActivity.java | 2 +-
9 files changed, 15 insertions(+), 15 deletions(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/src/jogl/classes/com/jogamp/opengl/util/GLPixelBuffer.java b/src/jogl/classes/com/jogamp/opengl/util/GLPixelBuffer.java
index b0fc7f332..2bd45e3e4 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/GLPixelBuffer.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/GLPixelBuffer.java
@@ -189,9 +189,9 @@ public class GLPixelBuffer {
private boolean disposed = false;
- public StringBuffer toString(StringBuffer sb) {
+ public StringBuilder toString(StringBuilder sb) {
if(null == sb) {
- sb = new StringBuffer();
+ sb = new StringBuilder();
}
sb.append(pixelAttributes).append(", dim ").append(width).append("x").append(height).append("x").append(depth).append(", pack ").append(pack)
.append(", disposed ").append(disposed).append(", valid ").append(isValid()).append(", buffer[sz [bytes ").append(byteSize).append(", elemSize ").append(bufferElemSize).append(", ").append(buffer).append("]");
diff --git a/src/jogl/classes/com/jogamp/opengl/util/awt/AWTGLPixelBuffer.java b/src/jogl/classes/com/jogamp/opengl/util/awt/AWTGLPixelBuffer.java
index aceb609a1..df1bbdf26 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/awt/AWTGLPixelBuffer.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/awt/AWTGLPixelBuffer.java
@@ -109,7 +109,7 @@ public class AWTGLPixelBuffer extends GLPixelBuffer {
super.dispose();
}
- public StringBuffer toString(StringBuffer sb) {
+ public StringBuilder toString(StringBuilder sb) {
sb = super.toString(sb);
sb.append(", allowRowStride ").append(allowRowStride).append(", image [").append(image.getWidth()).append("x").append(image.getHeight()).append(", ").append(image.toString()).append("]");
return sb;
@@ -225,4 +225,4 @@ public class AWTGLPixelBuffer extends GLPixelBuffer {
}
}
}
-}
\ No newline at end of file
+}
diff --git a/src/jogl/classes/com/jogamp/opengl/util/glsl/ShaderUtil.java b/src/jogl/classes/com/jogamp/opengl/util/glsl/ShaderUtil.java
index 066b2054d..d18fd4bae 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/glsl/ShaderUtil.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/glsl/ShaderUtil.java
@@ -249,10 +249,10 @@ public class ShaderUtil {
for(int i = source.length - 1; i>=0; i--) {
final CharSequence csq = source[i];
if(csq instanceof String) {
- // if ShaderCode.create(.. mutableStringBuffer == false )
+ // if ShaderCode.create(.. mutableStringBuilder == false )
tmp[i] = (String) csq;
} else {
- // if ShaderCode.create(.. mutableStringBuffer == true )
+ // if ShaderCode.create(.. mutableStringBuilder == true )
tmp[i] = source[i].toString();
}
}
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 6b41c0bc8..9f951d5da 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -75,7 +75,7 @@ import javax.media.opengl.GL;
// in case a fixed lookup function is being chosen, replace the name in our code
rsFp.replaceInShaderSource(myTextureLookupName, texLookupFuncName);
- // Cache the TextureSequence shader details in StringBuffer:
+ // Cache the TextureSequence shader details in StringBuilder:
final StringBuilder sFpIns = new StringBuilder();
// .. declaration of the texture sampler using the implementation specific type
@@ -217,4 +217,4 @@ public interface TextureSequence {
* @throws IllegalStateException if instance is not initialized
*/
public String getTextureLookupFragmentShaderImpl() throws IllegalStateException ;
-}
\ No newline at end of file
+}
diff --git a/src/jogl/classes/jogamp/opengl/windows/wgl/WGLGLCapabilities.java b/src/jogl/classes/jogamp/opengl/windows/wgl/WGLGLCapabilities.java
index 4444e9d63..bf2d3fa47 100644
--- a/src/jogl/classes/jogamp/opengl/windows/wgl/WGLGLCapabilities.java
+++ b/src/jogl/classes/jogamp/opengl/windows/wgl/WGLGLCapabilities.java
@@ -80,7 +80,7 @@ public class WGLGLCapabilities extends GLCapabilities {
public static final String PFD2String(PIXELFORMATDESCRIPTOR pfd, int pfdID) {
final int dwFlags = pfd.getDwFlags();
- StringBuffer sb = new StringBuffer();
+ StringBuilder sb = new StringBuilder();
boolean sep = false;
if( 0 != (GDI.PFD_DRAW_TO_WINDOW & dwFlags ) ) {
@@ -270,4 +270,4 @@ public class WGLGLCapabilities extends GLCapabilities {
sink.append(": ");
return super.toString(sink);
}
-}
\ No newline at end of file
+}
diff --git a/src/nativewindow/classes/javax/media/nativewindow/Capabilities.java b/src/nativewindow/classes/javax/media/nativewindow/Capabilities.java
index 8e83eda33..f2a8e2394 100644
--- a/src/nativewindow/classes/javax/media/nativewindow/Capabilities.java
+++ b/src/nativewindow/classes/javax/media/nativewindow/Capabilities.java
@@ -366,7 +366,7 @@ public class Capabilities implements CapabilitiesImmutable, Cloneable {
return msg.toString();
}
- /** Return a textual representation of this object's on/off screen state. Use the given StringBuffer [optional]. */
+ /** Return a textual representation of this object's on/off screen state. Use the given StringBuilder [optional]. */
protected StringBuilder onoffScreenToString(StringBuilder sink) {
if(null == sink) {
sink = new StringBuilder();
diff --git a/src/nativewindow/classes/javax/media/nativewindow/CapabilitiesImmutable.java b/src/nativewindow/classes/javax/media/nativewindow/CapabilitiesImmutable.java
index b801ab457..85659f286 100644
--- a/src/nativewindow/classes/javax/media/nativewindow/CapabilitiesImmutable.java
+++ b/src/nativewindow/classes/javax/media/nativewindow/CapabilitiesImmutable.java
@@ -130,7 +130,7 @@ public interface CapabilitiesImmutable extends VisualIDHolder, WriteCloneable, C
@Override
int hashCode();
- /** Return a textual representation of this object. Use the given StringBuffer [optional]. */
+ /** Return a textual representation of this object. Use the given StringBuilder [optional]. */
StringBuilder toString(StringBuilder sink);
/** Returns a textual representation of this object. */
diff --git a/src/newt/classes/com/jogamp/newt/MonitorMode.java b/src/newt/classes/com/jogamp/newt/MonitorMode.java
index dc7ae2c03..218cd8bd5 100644
--- a/src/newt/classes/com/jogamp/newt/MonitorMode.java
+++ b/src/newt/classes/com/jogamp/newt/MonitorMode.java
@@ -156,8 +156,8 @@ public class MonitorMode implements Comparable {
private final static String STR_DOUBLESCAN = "DoubleScan";
private final static String STR_SEP = ", ";
- public static final StringBuffer flags2String(int flags) {
- final StringBuffer sb = new StringBuffer();
+ public static final StringBuilder flags2String(int flags) {
+ final StringBuilder sb = new StringBuilder();
boolean sp = false;
if( 0 != ( flags & FLAG_INTERLACE ) ) {
sb.append(STR_INTERLACE);
diff --git a/src/newt/classes/jogamp/newt/driver/android/NewtVersionActivity.java b/src/newt/classes/jogamp/newt/driver/android/NewtVersionActivity.java
index de524d54c..9f6210269 100644
--- a/src/newt/classes/jogamp/newt/driver/android/NewtVersionActivity.java
+++ b/src/newt/classes/jogamp/newt/driver/android/NewtVersionActivity.java
@@ -78,7 +78,7 @@ public class NewtVersionActivity extends NewtBaseActivity {
glWindow.addGLEventListener(new GLEventListener() {
public void init(GLAutoDrawable drawable) {
GL gl = drawable.getGL();
- final StringBuffer sb = new StringBuffer();
+ final StringBuilder sb = new StringBuilder();
sb.append(JoglVersion.getGLInfo(gl, null, true)).append(Platform.NEWLINE);
sb.append("Requested: ").append(Platform.NEWLINE);
sb.append(drawable.getNativeSurface().getGraphicsConfiguration().getRequestedCapabilities()).append(Platform.NEWLINE).append(Platform.NEWLINE);
--
cgit v1.2.3
From 6332e13b2f0aa9818d37802302f04c90a4fa4239 Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Sat, 10 Aug 2013 09:14:19 +0200
Subject: GLMediaPlayer: Add multithreaded decoding w/ textureCount > 2 where
available EGL/FFMPeg. WIP!
Off-thread decoding:
If validated (impl) textureCount > 2, decoding happens on extra thread.
If decoding requires GL context, a shared context is created for decoding thread.
API Changes:
- initGLStream(..): Adds 'textureCount' as argument.
- TextureSequence.TexSeqEventListener.newFrameAvailable(..) exposes the new frame available
- TextureSequence.TextureFrame exposes the PTS (video)
Implementation:
- 'int validateTextureCount(int)': implementation decides whether textureCount can be > 2, i.e. off-thread decoding allowed,
default is NO w/ textureCount==2!
- 'boolean requiresOffthreadGLCtx()': implementation decides whether shared context is required for off-thread decoding
- 'syncFrame2Audio(TextureFrame frame)': implementation shall handle a/v sync, due to audio stream details (pts, buffered frames)
- FFMPEGMediaPlayer extends GLMediaPlayerImpl, no more EGLMediaPlayerImpl (redundant)
+++
- SyncedRingbuffer: Expose T[] array
+++
TODO:
- syncAV!
- test Android
---
make/scripts/tests.sh | 6 +-
.../com/jogamp/opengl/util/av/GLMediaPlayer.java | 35 +-
.../opengl/util/av/GLMediaPlayerFactory.java | 8 +-
.../opengl/util/texture/TextureSequence.java | 17 +-
.../android/av/AndroidGLMediaPlayerAPI14.java | 139 +++---
.../jogamp/opengl/util/av/EGLMediaPlayerImpl.java | 20 +-
.../jogamp/opengl/util/av/GLMediaPlayerImpl.java | 501 ++++++++++++++++-----
.../jogamp/opengl/util/av/NullGLMediaPlayer.java | 74 +--
.../jogamp/opengl/util/av/SyncedRingbuffer.java | 2 +
.../av/impl/FFMPEGDynamicLibraryBundleInfo.java | 16 +-
.../opengl/util/av/impl/FFMPEGMediaPlayer.java | 336 +++++---------
.../opengl/util/av/impl/OMXGLMediaPlayer.java | 44 +-
src/jogl/native/libav/ffmpeg_tool.h | 8 +
.../jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c | 59 ++-
src/jogl/native/openmax/omx_tool.c | 8 +-
.../jogl/demos/es2/TextureSequenceCubeES2.java | 2 +-
.../test/junit/jogl/demos/es2/av/MovieCube.java | 18 +-
.../test/junit/jogl/demos/es2/av/MovieSimple.java | 19 +-
18 files changed, 798 insertions(+), 514 deletions(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/make/scripts/tests.sh b/make/scripts/tests.sh
index 454bf21f3..ba4589d14 100644
--- a/make/scripts/tests.sh
+++ b/make/scripts/tests.sh
@@ -131,7 +131,7 @@ function jrun() {
#D_ARGS="-Djogl.1thread=true -Djogl.debug.Threading"
#D_ARGS="-Djogl.debug.DebugGL -Djogl.debug.TraceGL -Djogl.debug.GLContext.TraceSwitch -Djogl.debug=all"
#D_ARGS="-Djogamp.debug.IOUtil -Djogl.debug.GLSLCode -Djogl.debug.GLMediaPlayer"
- D_ARGS="-Djogl.debug.GLMediaPlayer -Djogl.debug.AudioSink"
+ #D_ARGS="-Djogl.debug.GLMediaPlayer -Djogl.debug.AudioSink"
#D_ARGS="-Djogl.debug.GLArrayData"
#D_ARGS="-Djogl.debug.GLDrawable"
#D_ARGS="-Djogl.debug.EGLDisplayUtil -Dnativewindow.debug.GraphicsConfiguration -Djogl.debug.GLDrawable"
@@ -314,8 +314,8 @@ function testawtswt() {
# av demos
#
#testnoawt jogamp.opengl.openal.av.ALDummyUsage $*
-#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
-testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
+testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
+#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
#
# core/newt (testnoawt and testawt)
diff --git a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
index 1825dbd47..a36bce305 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
@@ -40,12 +40,12 @@ import com.jogamp.opengl.util.texture.TextureSequence;
/**
* Lifecycle of an GLMediaPlayer:
*
- *
action
state before
state after
- *
{@link #initGLStream(GL, URLConnection)}
Uninitialized
Stopped
- *
{@link #start()}
Stopped, Paused
Playing
- *
{@link #stop()}
Playing, Paused
Stopped
- *
{@link #pause()}
Playing
Paused
- *
{@link #destroy(GL)}
ANY
Uninitialized
+ *
action
state before
state after
+ *
{@link #initGLStream(GL, int, URLConnection)}
Uninitialized
Stopped
+ *
{@link #start()}
Stopped, Paused
Playing
+ *
{@link #stop()}
Playing, Paused
Stopped
+ *
{@link #pause()}
Playing
Paused
+ *
{@link #destroy(GL)}
ANY
Uninitialized
*
*
* Current implementations (check each API doc link for details):
@@ -105,8 +105,12 @@ public interface GLMediaPlayer extends TextureSequence {
public int getTextureCount();
- /** Defaults to 0 */
+ /** Returns the texture target used by implementation. */
+ public int getTextureTarget();
+
+ /** Sets the texture unit. Defaults to 0. */
public void setTextureUnit(int u);
+
/** Sets the texture min-mag filter, defaults to {@link GL#GL_NEAREST}. */
public void setTextureMinMagFilter(int[] minMagFilter);
/** Sets the texture min-mag filter, defaults to {@link GL#GL_CLAMP_TO_EDGE}. */
@@ -119,6 +123,7 @@ public interface GLMediaPlayer extends TextureSequence {
* Uninitialized -> Stopped
*
* @param gl current GL object. If null, no video output and textures will be available.
+ * @param textureCount desired number of buffered textures to be decoded off-thread, use 1 for on-thread decoding.
* @param urlConn the stream connection
* @return the new state
*
@@ -126,7 +131,7 @@ public interface GLMediaPlayer extends TextureSequence {
* @throws IOException in case of difficulties to open or process the stream
* @throws GLException in case of difficulties to initialize the GL resources
*/
- public State initGLStream(GL gl, URLConnection urlConn) throws IllegalStateException, GLException, IOException;
+ public State initGLStream(GL gl, int textureCount, URLConnection urlConn) throws IllegalStateException, GLException, IOException;
/**
* Releases the GL and stream resources.
@@ -161,10 +166,20 @@ public interface GLMediaPlayer extends TextureSequence {
public State getState();
/**
- * @return time current position in milliseconds
+ * @return current streaming position in milliseconds
**/
public int getCurrentPosition();
+ /**
+ * @return current video PTS in milliseconds of {@link #getLastTexture()}
+ **/
+ public int getVideoPTS();
+
+ /**
+ * @return current audio PTS in milliseconds.
+ **/
+ public int getAudioPTS();
+
/**
* Allowed in state Stopped, Playing and Paused, otherwise ignored.
*
@@ -187,7 +202,7 @@ public interface GLMediaPlayer extends TextureSequence {
*
*
* @see #addEventListener(GLMediaEventListener)
- * @see GLMediaEventListener#newFrameAvailable(GLMediaPlayer, long)
+ * @see GLMediaEventListener#newFrameAvailable(GLMediaPlayer, TextureFrame, long)
*/
@Override
public TextureSequence.TextureFrame getNextTexture(GL gl, boolean blocking) throws IllegalStateException;
diff --git a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
index f09531f7f..2707dd6d2 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
@@ -29,18 +29,20 @@ package com.jogamp.opengl.util.av;
import jogamp.opengl.util.av.NullGLMediaPlayer;
-import com.jogamp.common.os.AndroidVersion;
-import com.jogamp.common.os.Platform;
import com.jogamp.common.util.ReflectionUtil;
public class GLMediaPlayerFactory {
private static final String AndroidGLMediaPlayerAPI14ClazzName = "jogamp.opengl.android.av.AndroidGLMediaPlayerAPI14";
private static final String FFMPEGMediaPlayerClazzName = "jogamp.opengl.util.av.impl.FFMPEGMediaPlayer";
+ private static final String OMXGLMediaPlayerClazzName = "jogamp.opengl.util.av.impl.OMXGLMediaPlayer";
private static final String isAvailableMethodName = "isAvailable";
public static GLMediaPlayer createDefault() {
final ClassLoader cl = GLMediaPlayerFactory.class.getClassLoader();
- GLMediaPlayer sink = create(cl, AndroidGLMediaPlayerAPI14ClazzName);
+ GLMediaPlayer sink = create(cl, OMXGLMediaPlayerClazzName);
+ if( null == sink ) {
+ sink = create(cl, AndroidGLMediaPlayerAPI14ClazzName);
+ }
if( null == sink ) {
sink = create(cl, FFMPEGMediaPlayerClazzName);
}
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 9f951d5da..3f739b2cc 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -112,26 +112,35 @@ public interface TextureSequence {
public static class TextureFrame {
public TextureFrame(Texture t) {
texture = t;
+ pts = 0;
}
public final Texture getTexture() { return texture; }
+ public final int getPTS() { return pts; }
+ public final void setPTS(int pts) { this.pts = pts; }
public String toString() {
- return "TextureFrame[" + texture + "]";
+ return "TextureFrame[" + pts + "ms: " + texture + "]";
}
protected final Texture texture;
+ protected int pts;
}
public interface TexSeqEventListener {
/**
- * Signaling listeners that {@link TextureSequence#getNextTexture(GL, boolean)} is able to deliver a new frame.
+ * Signaling listeners that a new {@link TextureFrame} is available.
+ *
+ * User shall utilize {@link TextureSequence#getNextTexture(GL, boolean)} to dequeue it to maintain
+ * a consistent queue.
+ *
* @param ts the event source
+ * @param newFrame the newly enqueued frame
* @param when system time in msec.
**/
- public void newFrameAvailable(T ts, long when);
+ public void newFrameAvailable(T ts, TextureFrame newFrame, long when);
}
- /** Return the texture unit to be used with this frame. */
+ /** Return the texture unit used to render the current frame. */
public int getTextureUnit();
public int[] getTextureMinMagFilter();
diff --git a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
index 8356a2bae..765cda084 100644
--- a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
+++ b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
@@ -28,12 +28,14 @@
package jogamp.opengl.android.av;
import java.io.IOException;
+import java.nio.Buffer;
import javax.media.opengl.GL;
import javax.media.opengl.GLES2;
import com.jogamp.common.os.AndroidVersion;
import com.jogamp.common.os.Platform;
+import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureSequence;
import jogamp.common.os.android.StaticContext;
@@ -76,7 +78,6 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
MediaPlayer mp;
volatile boolean updateSurface = false;
Object updateSurfaceLock = new Object();
- TextureSequence.TextureFrame lastTexFrame = null;
/**
private static String toString(MediaPlayer m) {
@@ -90,17 +91,16 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
throw new RuntimeException("AndroidGLMediaPlayerAPI14 not available");
}
this.setTextureTarget(GLES2.GL_TEXTURE_EXTERNAL_OES);
- this.setTextureCount(1);
mp = new MediaPlayer();
}
@Override
- protected boolean setPlaySpeedImpl(float rate) {
+ protected final boolean setPlaySpeedImpl(float rate) {
return false;
}
@Override
- protected boolean startImpl() {
+ protected final boolean startImpl() {
if(null != mp) {
try {
mp.start();
@@ -115,7 +115,7 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected boolean pauseImpl() {
+ protected final boolean pauseImpl() {
if(null != mp) {
wakeUp(false);
try {
@@ -131,7 +131,7 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected boolean stopImpl() {
+ protected final boolean stopImpl() {
if(null != mp) {
wakeUp(false);
try {
@@ -147,7 +147,7 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected int seekImpl(int msec) {
+ protected final int seekImpl(int msec) {
if(null != mp) {
mp.seekTo(msec);
return mp.getCurrentPosition();
@@ -155,40 +155,6 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
return 0;
}
- @Override
- protected TextureSequence.TextureFrame getLastTextureImpl() {
- return lastTexFrame;
- }
-
- @Override
- protected TextureSequence.TextureFrame getNextTextureImpl(GL gl, boolean blocking) {
- if(null != stex && null != mp) {
- // Only block once, no while-loop.
- // This relaxes locking code of non crucial resources/events.
- boolean update = updateSurface;
- if(!update && blocking) {
- synchronized(updateSurfaceLock) {
- if(!updateSurface) { // volatile OK.
- try {
- updateSurfaceLock.wait();
- } catch (InterruptedException e) {
- e.printStackTrace();
- }
- }
- updateSurface = false;
- update = true;
- }
- }
- if(update) {
- stex.updateTexImage();
- // stex.getTransformMatrix(atex.getSTMatrix());
- lastTexFrame=texFrames[0];
- }
-
- }
- return lastTexFrame;
- }
-
private void wakeUp(boolean newFrame) {
synchronized(updateSurfaceLock) {
if(newFrame) {
@@ -199,12 +165,13 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected int getCurrentPositionImpl() {
- return null != mp ? mp.getCurrentPosition() : 0;
- }
+ protected final int getCurrentPositionImpl() { return null != mp ? mp.getCurrentPosition() : 0; }
+
+ @Override
+ protected final int getAudioPTSImpl() { return getCurrentPositionImpl(); }
@Override
- protected void destroyImpl(GL gl) {
+ protected final void destroyImpl(GL gl) {
if(null != mp) {
wakeUp(false);
mp.release();
@@ -213,9 +180,25 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
SurfaceTexture stex = null;
+ public static class SurfaceTextureFrame extends TextureSequence.TextureFrame {
+ public SurfaceTextureFrame(Texture t, SurfaceTexture stex) {
+ super(t);
+ this.surfaceTex = stex;
+ this.surface = new Surface(stex);
+ }
+
+ public final SurfaceTexture getSurfaceTexture() { return surfaceTex; }
+ public final Surface getSurface() { return surface; }
+
+ public String toString() {
+ return "SurfaceTextureFrame[" + pts + "ms: " + texture + ", " + surfaceTex + "]";
+ }
+ private final SurfaceTexture surfaceTex;
+ private final Surface surface;
+ }
@Override
- protected void initGLStreamImpl(GL gl, int[] texNames) throws IOException {
+ protected final void initGLStreamImpl(GL gl) throws IOException {
if(null!=mp && null!=urlConn) {
try {
final Uri uri = Uri.parse(urlConn.getURL().toExternalForm());
@@ -227,44 +210,86 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
} catch (IllegalStateException e) {
throw new RuntimeException(e);
}
- stex = new SurfaceTexture(texNames[0]); // only 1 texture
- stex.setOnFrameAvailableListener(onFrameAvailableListener);
+ if( null == stex ) {
+ throw new InternalError("XXX");
+ }
final Surface surf = new Surface(stex);
mp.setSurface(surf);
surf.release();
+ mp.setSurface(null);
try {
mp.prepare();
} catch (IOException ioe) {
throw new IOException("MediaPlayer failed to process stream <"+urlConn.getURL().toExternalForm()+">: "+ioe.getMessage(), ioe);
}
+ final String icodec = "android";
updateAttributes(mp.getVideoWidth(), mp.getVideoHeight(),
0, 0, 0,
0f, 0, mp.getDuration(),
- null, null);
+ icodec, icodec);
+ }
+ }
+
+ @Override
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ if(null != stex && null != mp) {
+ final SurfaceTextureFrame nextSFrame = (SurfaceTextureFrame) nextFrame;
+ final Surface nextSurface = nextSFrame.getSurface();
+ mp.setSurface(nextSurface);
+ nextSurface.release();
+
+ // Only block once, no while-loop.
+ // This relaxes locking code of non crucial resources/events.
+ boolean update = updateSurface;
+ if(!update && blocking) {
+ synchronized(updateSurfaceLock) {
+ if(!updateSurface) { // volatile OK.
+ try {
+ updateSurfaceLock.wait();
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ update = updateSurface;
+ updateSurface = false;
+ }
+ }
+ if(update) {
+ final SurfaceTexture nextSTex = nextSFrame.getSurfaceTexture();
+ nextSTex.updateTexImage();
+ // nextFrame.setPTS( (int) ( nextSTex.getTimestamp() / 1000000L ) ); // nano -9 -> milli -3
+ nextFrame.setPTS( mp.getCurrentPosition() );
+ // stex.getTransformMatrix(atex.getSTMatrix());
+ }
}
+ return true;
}
+ @Override
+ protected final void syncFrame2Audio(TextureFrame frame) {}
@Override
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- lastTexFrame = new TextureSequence.TextureFrame( createTexImageImpl(gl, idx, tex, width, height, true) );
- return lastTexFrame;
+ protected final TextureSequence.TextureFrame createTexImage(GL gl, int texName) {
+ if( null != stex ) {
+ throw new InternalError("XXX");
+ }
+ stex = new SurfaceTexture(texName); // only 1 texture
+ stex.setOnFrameAvailableListener(onFrameAvailableListener);
+ return new TextureSequence.TextureFrame( createTexImageImpl(gl, texName, width, height, true) );
}
@Override
- protected void destroyTexImage(GL gl, TextureSequence.TextureFrame imgTex) {
+ protected final void destroyTexFrame(GL gl, TextureSequence.TextureFrame imgTex) {
if(null != stex) {
stex.release();
stex = null;
}
- lastTexFrame = null;
- super.destroyTexImage(gl, imgTex);
+ super.destroyTexFrame(gl, imgTex);
}
protected OnFrameAvailableListener onFrameAvailableListener = new OnFrameAvailableListener() {
@Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
wakeUp(true);
- AndroidGLMediaPlayerAPI14.this.newFrameAvailable();
}
- };
+ };
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java b/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
index 274ccffd5..57d5ff625 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
@@ -77,19 +77,19 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
}
- protected EGLMediaPlayerImpl() {
- this(TextureType.GL, false);
- }
-
protected EGLMediaPlayerImpl(TextureType texType, boolean useKHRSync) {
super();
this.texType = texType;
this.useKHRSync = useKHRSync;
}
+ @Override
+ protected final int validateTextureCount(int desiredTextureCount) {
+ return desiredTextureCount>1 ? desiredTextureCount : 2;
+ }
@Override
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- final Texture texture = super.createTexImageImpl(gl, idx, tex, width, height, false);
+ protected TextureSequence.TextureFrame createTexImage(GL gl, int texName) {
+ final Texture texture = super.createTexImageImpl(gl, texName, width, height, false);
final Buffer clientBuffer;
final long image;
final long sync;
@@ -117,7 +117,7 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
EGLExt.EGL_GL_TEXTURE_2D_KHR,
clientBuffer, nioTmp);
if (0==image) {
- throw new RuntimeException("EGLImage creation failed: "+EGL.eglGetError()+", ctx "+eglCtx+", tex "+tex[idx]+", err "+toHexString(EGL.eglGetError()));
+ throw new RuntimeException("EGLImage creation failed: "+EGL.eglGetError()+", ctx "+eglCtx+", tex "+texName+", err "+toHexString(EGL.eglGetError()));
}
} else {
clientBuffer = null;
@@ -141,7 +141,7 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
}
@Override
- protected void destroyTexImage(GL gl, TextureSequence.TextureFrame imgTex) {
+ protected void destroyTexFrame(GL gl, TextureSequence.TextureFrame frame) {
final boolean eglUsage = TextureType.KHRImage == texType || useKHRSync ;
final EGLContext eglCtx;
final EGLExt eglExt;
@@ -156,7 +156,7 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
eglExt = null;
eglDrawable = null;
}
- final EGLTextureFrame eglTex = (EGLTextureFrame) imgTex;
+ final EGLTextureFrame eglTex = (EGLTextureFrame) frame;
if(0!=eglTex.getImage()) {
eglExt.eglDestroyImageKHR(eglDrawable.getNativeSurface().getDisplayHandle(), eglTex.getImage());
@@ -164,6 +164,6 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
if(0!=eglTex.getSync()) {
eglExt.eglDestroySyncKHR(eglDrawable.getNativeSurface().getDisplayHandle(), eglTex.getSync());
}
- super.destroyTexImage(gl, imgTex);
+ super.destroyTexFrame(gl, frame);
}
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
index 2ff91a3f6..bc297dc21 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
@@ -30,13 +30,17 @@ package jogamp.opengl.util.av;
import java.io.IOException;
import java.net.URLConnection;
import java.util.ArrayList;
-import java.util.HashMap;
import java.util.Iterator;
+import javax.media.nativewindow.AbstractGraphicsDevice;
import javax.media.opengl.GL;
import javax.media.opengl.GL2;
+import javax.media.opengl.GLContext;
+import javax.media.opengl.GLDrawable;
+import javax.media.opengl.GLDrawableFactory;
import javax.media.opengl.GLES2;
import javax.media.opengl.GLException;
+import javax.media.opengl.GLProfile;
import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
@@ -45,7 +49,7 @@ import com.jogamp.opengl.util.texture.TextureSequence;
/**
* After object creation an implementation may customize the behavior:
*
@@ -59,6 +63,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected static final String unknown = "unknown";
protected State state;
+
protected int textureCount;
protected int textureTarget;
protected int textureFormat;
@@ -74,35 +79,38 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected volatile float playSpeed = 1.0f;
- /** Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int width = 0;
- /** Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int height = 0;
- /** Video fps. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Video fps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected float fps = 0;
- /** Stream bps. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Stream bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int bps_stream = 0;
- /** Video bps. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Video bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int bps_video = 0;
- /** Audio bps. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Audio bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int bps_audio = 0;
- /** In frames. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** In frames. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int totalFrames = 0;
- /** In ms. Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** In ms. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected int duration = 0;
- /** Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected String acodec = unknown;
- /** Shall be set by the {@link #initGLStreamImpl(GL, int[])} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
protected String vcodec = unknown;
protected int frameNumber = 0;
+ protected int currentVideoPTS = 0;
- protected TextureSequence.TextureFrame[] texFrames = null;
- protected HashMap texFrameMap = new HashMap();
+ protected SyncedRingbuffer videoFramesFree = null;
+ protected SyncedRingbuffer videoFramesDecoded = null;
+ protected volatile TextureFrame lastFrame = null;
+
private ArrayList eventListeners = new ArrayList();
protected GLMediaPlayerImpl() {
- this.textureCount=3;
+ this.textureCount=0;
this.textureTarget=GL.GL_TEXTURE_2D;
this.textureFormat = GL.GL_RGBA;
this.textureInternalFormat = GL.GL_RGBA;
@@ -112,14 +120,14 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
@Override
- public void setTextureUnit(int u) { texUnit = u; }
+ public final void setTextureUnit(int u) { texUnit = u; }
@Override
- public int getTextureUnit() { return texUnit; }
+ public final int getTextureUnit() { return texUnit; }
+
+ @Override
+ public final int getTextureTarget() { return textureTarget; }
- protected final void setTextureCount(int textureCount) {
- this.textureCount=textureCount;
- }
@Override
public final int getTextureCount() { return textureCount; }
@@ -134,29 +142,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
public final int[] getTextureMinMagFilter() { return texMinMagFilter; }
public final void setTextureWrapST(int[] wrapST) { texWrapST[0] = wrapST[0]; texWrapST[1] = wrapST[1];}
- public final int[] getTextureWrapST() { return texWrapST; }
-
- @Override
- public final TextureSequence.TextureFrame getLastTexture() throws IllegalStateException {
- if(State.Uninitialized == state) {
- throw new IllegalStateException("Instance not initialized: "+this);
- }
- return getLastTextureImpl();
- }
- protected abstract TextureSequence.TextureFrame getLastTextureImpl();
-
- @Override
- public final synchronized TextureSequence.TextureFrame getNextTexture(GL gl, boolean blocking) throws IllegalStateException {
- if(State.Uninitialized == state) {
- throw new IllegalStateException("Instance not initialized: "+this);
- }
- if(State.Playing == state) {
- final TextureSequence.TextureFrame f = getNextTextureImpl(gl, blocking);
- return f;
- }
- return getLastTextureImpl();
- }
- protected abstract TextureSequence.TextureFrame getNextTextureImpl(GL gl, boolean blocking);
+ public final int[] getTextureWrapST() { return texWrapST; }
@Override
public String getRequiredExtensionsShaderStub() throws IllegalStateException {
@@ -229,12 +215,15 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected abstract boolean setPlaySpeedImpl(float rate);
public final State start() {
- switch(state) {
+ switch( state ) {
case Stopped:
+ /** fall-through intended */
case Paused:
- if(startImpl()) {
+ if( startImpl() ) {
+ resumeFramePusher();
state = State.Playing;
}
+ default:
}
if(DEBUG) { System.err.println("Start: "+toString()); }
return state;
@@ -242,7 +231,8 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected abstract boolean startImpl();
public final State pause() {
- if(State.Playing == state && pauseImpl()) {
+ if( State.Playing == state && pauseImpl() ) {
+ pauseFramePusher();
state = State.Paused;
}
if(DEBUG) { System.err.println("Pause: "+toString()); }
@@ -251,12 +241,15 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected abstract boolean pauseImpl();
public final State stop() {
- switch(state) {
+ switch( state ) {
case Playing:
+ /** fall-through intended */
case Paused:
- if(stopImpl()) {
+ if( stopImpl() ) {
+ pauseFramePusher();
state = State.Stopped;
}
+ default:
}
if(DEBUG) { System.err.println("Stop: "+toString()); }
return state;
@@ -265,61 +258,70 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
@Override
public final int getCurrentPosition() {
- if(State.Uninitialized != state) {
+ if( State.Uninitialized != state ) {
return getCurrentPositionImpl();
}
return 0;
}
protected abstract int getCurrentPositionImpl();
+ @Override
+ public final int getVideoPTS() { return currentVideoPTS; }
+
+ @Override
+ public final int getAudioPTS() {
+ if( State.Uninitialized != state ) {
+ return getAudioPTSImpl();
+ }
+ return 0;
+ }
+ protected abstract int getAudioPTSImpl();
+
public final int seek(int msec) {
- final int cp;
+ final int pts1;
switch(state) {
case Stopped:
case Playing:
case Paused:
- cp = seekImpl(msec);
+ pauseFramePusher();
+ pts1 = seekImpl(msec);
+ currentVideoPTS=pts1;
+ resumeFramePusher();
break;
default:
- cp = 0;
+ pts1 = 0;
}
if(DEBUG) { System.err.println("Seek("+msec+"): "+toString()); }
- return cp;
+ return pts1;
}
protected abstract int seekImpl(int msec);
public final State getState() { return state; }
@Override
- public final State initGLStream(GL gl, URLConnection urlConn) throws IllegalStateException, GLException, IOException {
+ public final State initGLStream(GL gl, int reqTextureCount, URLConnection urlConn) throws IllegalStateException, GLException, IOException {
if(State.Uninitialized != state) {
throw new IllegalStateException("Instance not in state "+State.Uninitialized+", but "+state+", "+this);
}
this.urlConn = urlConn;
if (this.urlConn != null) {
try {
- if(null != gl) {
- if(null!=texFrames) {
- // re-init ..
- removeAllImageTextures(gl);
- } else {
- texFrames = new TextureSequence.TextureFrame[textureCount];
- }
- final int[] tex = new int[textureCount];
- {
- gl.glGenTextures(textureCount, tex, 0);
- final int err = gl.glGetError();
- if( GL.GL_NO_ERROR != err ) {
- throw new RuntimeException("TextureNames creation failed (num: "+textureCount+"): err "+toHexString(err));
- }
+ if( null != gl ) {
+ removeAllTextureFrames(gl);
+ textureCount = validateTextureCount(reqTextureCount);
+ if( textureCount < 2 ) {
+ throw new InternalError("Validated texture count < 2: "+textureCount);
}
- initGLStreamImpl(gl, tex);
-
- for(int i=0; i(createTexFrames(gl, textureCount), true /* full */);
+ if( 2 < textureCount ) {
+ videoFramesDecoded = new SyncedRingbuffer(new TextureFrame[textureCount], false /* full */);
+ framePusher = new FramePusher(gl, requiresOffthreadGLCtx());
+ framePusher.doStart();
+ } else {
+ videoFramesDecoded = null;
}
+ lastFrame = videoFramesFree.getBlocking(false /* clearRef */ );
}
state = State.Stopped;
return state;
@@ -329,35 +331,42 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
return state;
}
+ /**
+ * Returns the validated number of textures to be handled.
+ *
+ * Default is always 2 textures, last texture and the decoding texture.
+ *
+ */
+ protected int validateTextureCount(int desiredTextureCount) {
+ return 2;
+ }
+ protected boolean requiresOffthreadGLCtx() { return false; }
- /**
- * Implementation shall set the following set of data here
- * @param gl TODO
- * @param texNames TODO
- * @see #width
- * @see #height
- * @see #fps
- * @see #bps_stream
- * @see #totalFrames
- * @see #acodec
- * @see #vcodec
- */
- protected abstract void initGLStreamImpl(GL gl, int[] texNames) throws IOException;
-
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- return new TextureSequence.TextureFrame( createTexImageImpl(gl, idx, tex, width, height, false) );
+ private final TextureFrame[] createTexFrames(GL gl, final int count) {
+ final int[] texNames = new int[count];
+ gl.glGenTextures(count, texNames, 0);
+ final int err = gl.glGetError();
+ if( GL.GL_NO_ERROR != err ) {
+ throw new RuntimeException("TextureNames creation failed (num: "+count+"): err "+toHexString(err));
+ }
+ final TextureFrame[] texFrames = new TextureFrame[count];
+ for(int i=0; i tex[idx] ) {
- throw new RuntimeException("TextureName "+toHexString(tex[idx])+" invalid.");
+ protected final Texture createTexImageImpl(GL gl, int texName, int tWidth, int tHeight, boolean mustFlipVertically) {
+ if( 0 > texName ) {
+ throw new RuntimeException("TextureName "+toHexString(texName)+" invalid.");
}
gl.glActiveTexture(GL.GL_TEXTURE0+getTextureUnit());
- gl.glBindTexture(textureTarget, tex[idx]);
+ gl.glBindTexture(textureTarget, texName);
{
final int err = gl.glGetError();
if( GL.GL_NO_ERROR != err ) {
- throw new RuntimeException("Couldn't bind textureName "+toHexString(tex[idx])+" to 2D target, err "+toHexString(err));
+ throw new RuntimeException("Couldn't bind textureName "+toHexString(texName)+" to 2D target, err "+toHexString(err));
}
}
@@ -389,30 +398,297 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
gl.glTexParameteri(textureTarget, GL.GL_TEXTURE_WRAP_S, texWrapST[0]);
gl.glTexParameteri(textureTarget, GL.GL_TEXTURE_WRAP_T, texWrapST[1]);
- return com.jogamp.opengl.util.texture.TextureIO.newTexture(tex[idx],
- textureTarget,
+ return com.jogamp.opengl.util.texture.TextureIO.newTexture(
+ texName, textureTarget,
tWidth, tHeight,
width, height,
mustFlipVertically);
}
+
+ private final void removeAllTextureFrames(GL gl) {
+ if( null != videoFramesFree ) {
+ final TextureFrame[] texFrames = videoFramesFree.getArray();
+ videoFramesFree = null;
+ videoFramesDecoded = null;
+ lastFrame = null;
+ for(int i=0; i i = eventListeners.iterator(); i.hasNext(); ) {
+ i.next().newFrameAvailable(this, frame, System.currentTimeMillis());
+ }
+ }
+ }
+
+ class FramePusher extends Thread {
+ private volatile boolean isRunning = false;
+ private volatile boolean isActive = false;
+
+ private volatile boolean shallPause = true;
+ private volatile boolean shallStop = false;
+
+ private final GL gl;
+ private GLDrawable dummyDrawable = null;
+ private GLContext sharedGLCtx = null;
+
+ FramePusher(GL gl, boolean createSharedCtx) {
+ setDaemon(true);
+ this.gl = createSharedCtx ? createSharedGL(gl) : gl;
+ }
+
+ private GL createSharedGL(GL gl) {
+ final GLContext glCtx = gl.getContext();
+ final boolean glCtxCurrent = glCtx.isCurrent();
+ final GLProfile glp = gl.getGLProfile();
+ final GLDrawableFactory factory = GLDrawableFactory.getFactory(glp);
+ final AbstractGraphicsDevice device = glCtx.getGLDrawable().getNativeSurface().getGraphicsConfiguration().getScreen().getDevice();
+ dummyDrawable = factory.createDummyDrawable(device, true, glp); // own device!
+ dummyDrawable.setRealized(true);
+ sharedGLCtx = dummyDrawable.createContext(glCtx);
+ makeCurrent(sharedGLCtx);
+ if( glCtxCurrent ) {
+ makeCurrent(glCtx);
+ } else {
+ sharedGLCtx.release();
+ }
+ return sharedGLCtx.getGL();
+ }
+ private void makeCurrent(GLContext ctx) {
+ if( GLContext.CONTEXT_NOT_CURRENT >= ctx.makeCurrent() ) {
+ throw new GLException("Couldn't make ctx current: "+ctx);
+ }
+ }
+
+ private void destroySharedGL() {
+ if( null != sharedGLCtx ) {
+ if( sharedGLCtx.isCreated() ) {
+ // Catch dispose GLExceptions by GLEventListener, just 'print' them
+ // so we can continue with the destruction.
+ try {
+ sharedGLCtx.destroy();
+ } catch (GLException gle) {
+ gle.printStackTrace();
+ }
+ }
+ sharedGLCtx = null;
+ }
+ if( null != dummyDrawable ) {
+ final AbstractGraphicsDevice device = dummyDrawable.getNativeSurface().getGraphicsConfiguration().getScreen().getDevice();
+ dummyDrawable.setRealized(false);
+ dummyDrawable = null;
+ device.close();
+ }
+ }
+
+ public synchronized void doPause() {
+ if( isActive ) {
+ shallPause = true;
+ while( isActive ) {
+ try {
+ this.wait(); // wait until paused
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+ }
+ public synchronized void doResume() {
+ if( isRunning && !isActive ) {
+ shallPause = false;
+ while( !isActive ) {
+ this.notify(); // wake-up pause-block
+ try {
+ this.wait(); // wait until resumed
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+ }
+ public synchronized void doStart() {
+ start();
+ while( !isRunning ) {
+ try {
+ this.wait(); // wait until started
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+ public synchronized void doStop() {
+ if( isRunning ) {
+ shallStop = true;
+ while( isRunning ) {
+ this.notify(); // wake-up pause-block (opt)
+ try {
+ this.wait(); // wait until stopped
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+ }
+ public boolean isRunning() { return isRunning; }
+ public boolean isActive() { return isActive; }
+
+ public void run() {
+ setName(getName()+"-FramePusher_"+FramePusherInstanceId);
+ FramePusherInstanceId++;
+
+ synchronized ( this ) {
+ if( null != sharedGLCtx ) {
+ makeCurrent( sharedGLCtx );
+ }
+ isRunning = true;
+ this.notify(); // wake-up doStart()
+ }
+
+ while( !shallStop ){
+ if( shallPause ) {
+ synchronized ( this ) {
+ while( shallPause && !shallStop ) {
+ isActive = false;
+ this.notify(); // wake-up doPause()
+ try {
+ this.wait(); // wait until resumed
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+ isActive = true;
+ this.notify(); // wake-up doResume()
+ }
+ }
+
+ if( !shallStop ) {
+ TextureFrame nextFrame = null;
+ boolean ok = false;
+ try {
+ nextFrame = videoFramesFree.getBlocking(true /* clearRef */ );
+ if( getNextTextureImpl(gl, nextFrame, true) ) {
+ gl.glFinish();
+ videoFramesDecoded.putBlocking(nextFrame);
+ newFrameAvailable(nextFrame);
+ ok = true;
+ }
+ } catch (InterruptedException e) {
+ if( !shallStop && !shallPause ) {
+ e.printStackTrace(); // oops
+ shallPause = false;
+ shallStop = true;
+ }
+ } finally {
+ if( !ok && null != nextFrame ) { // put back
+ videoFramesFree.put(nextFrame);
+ }
+ }
+ }
+ }
+ destroySharedGL();
+ synchronized ( this ) {
+ isRunning = false;
+ isActive = false;
+ this.notify(); // wake-up doStop()
+ }
+ }
+ }
+ static int FramePusherInstanceId = 0;
+ private FramePusher framePusher = null;
+ private final void pauseFramePusher() {
+ if( null != framePusher ) {
+ framePusher.doPause();
+ }
+ }
+ private final void resumeFramePusher() {
+ if( null != framePusher ) {
+ framePusher.doResume();
+ }
+ }
+ private final void destroyFramePusher() {
+ if( null != framePusher ) {
+ framePusher.doStop();
+ framePusher = null;
+ }
+ }
+
protected final void updateAttributes(int width, int height, int bps_stream, int bps_video, int bps_audio,
float fps, int totalFrames, int duration,
String vcodec, String acodec) {
@@ -458,19 +734,12 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
}
}
- protected final void newFrameAvailable() {
- frameNumber++;
- synchronized(eventListenersLock) {
- for(Iterator i = eventListeners.iterator(); i.hasNext(); ) {
- i.next().newFrameAvailable(this, System.currentTimeMillis());
- }
- }
- }
@Override
public final synchronized State destroy(GL gl) {
+ destroyFramePusher();
destroyImpl(gl);
- removeAllImageTextures(gl);
+ removeAllTextureFrames(gl);
state = State.Uninitialized;
return state;
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
index cd48c3962..f1ce42257 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
@@ -49,66 +49,68 @@ import com.jogamp.opengl.util.texture.TextureSequence;
*/
public class NullGLMediaPlayer extends GLMediaPlayerImpl {
private TextureData texData = null;
- private TextureSequence.TextureFrame frame = null;
private int pos_ms = 0;
private int pos_start = 0;
public NullGLMediaPlayer() {
super();
- this.setTextureCount(1);
}
@Override
- protected boolean setPlaySpeedImpl(float rate) {
+ protected final boolean setPlaySpeedImpl(float rate) {
return false;
}
@Override
- protected boolean startImpl() {
+ protected final boolean startImpl() {
pos_start = (int)System.currentTimeMillis();
return true;
}
@Override
- protected boolean pauseImpl() {
+ protected final boolean pauseImpl() {
return true;
}
@Override
- protected boolean stopImpl() {
+ protected final boolean stopImpl() {
return true;
}
@Override
- protected int seekImpl(int msec) {
+ protected final int seekImpl(int msec) {
pos_ms = msec;
validatePos();
return pos_ms;
}
@Override
- protected TextureSequence.TextureFrame getLastTextureImpl() {
- return frame;
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ return true;
}
-
@Override
- protected TextureSequence.TextureFrame getNextTextureImpl(GL gl, boolean blocking) {
- return frame;
- }
+ protected final void syncFrame2Audio(TextureFrame frame) { }
@Override
- protected int getCurrentPositionImpl() {
+ protected final int getCurrentPositionImpl() {
pos_ms = (int)System.currentTimeMillis() - pos_start;
validatePos();
return pos_ms;
}
+ @Override
+ protected final int getAudioPTSImpl() { return getCurrentPositionImpl(); }
+
@Override
- protected void destroyImpl(GL gl) {
+ protected final void destroyImpl(GL gl) {
+ if(null != texData) {
+ texData.destroy();
+ texData = null;
+ }
}
-
+
@Override
- protected void initGLStreamImpl(GL gl, int[] texNames) throws IOException {
+ protected final void initGLStreamImpl(GL gl) throws IOException {
try {
URLConnection urlConn = IOUtil.getResource("jogl/util/data/av/test-ntsc01-160x90.png", this.getClass().getClassLoader());
if(null != urlConn) {
@@ -117,44 +119,44 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
} catch (Exception e) {
e.printStackTrace();
}
+ final int _w, _h;
if(null != texData) {
- width = texData.getWidth();
- height = texData.getHeight();
+ _w = texData.getWidth();
+ _h = texData.getHeight();
} else {
- width = 640;
- height = 480;
- ByteBuffer buffer = Buffers.newDirectByteBuffer(width*height*4);
+ _w = 640;
+ _h = 480;
+ ByteBuffer buffer = Buffers.newDirectByteBuffer(_w*_h*4);
while(buffer.hasRemaining()) {
buffer.put((byte) 0xEA); buffer.put((byte) 0xEA); buffer.put((byte) 0xEA); buffer.put((byte) 0xEA);
}
buffer.rewind();
texData = new TextureData(GLProfile.getGL2ES2(),
- GL.GL_RGBA, width, height, 0,
+ GL.GL_RGBA, _w, _h, 0,
GL.GL_RGBA, GL.GL_UNSIGNED_BYTE, false,
false, false, buffer, null);
}
- fps = 24f;
- duration = 10*60*1000; // msec
- totalFrames = (int) ( (duration/1000)*fps );
- vcodec = "png-static";
+ final float _fps = 24f;
+ final int _duration = 10*60*1000; // msec
+ final int _totalFrames = (int) ( (_duration/1000)*_fps );
+ updateAttributes(_w, _h,
+ 0, 0, 0,
+ _fps, _totalFrames, _duration,
+ "png-static", null);
}
@Override
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- Texture texture = super.createTexImageImpl(gl, idx, tex, width, height, false);
+ protected final TextureSequence.TextureFrame createTexImage(GL gl, int texName) {
+ final Texture texture = super.createTexImageImpl(gl, texName, width, height, false);
if(null != texData) {
texture.updateImage(gl, texData);
- texData.destroy();
- texData = null;
}
- frame = new TextureSequence.TextureFrame( texture );
- return frame;
+ return new TextureSequence.TextureFrame( texture );
}
@Override
- protected void destroyTexImage(GL gl, TextureSequence.TextureFrame imgTex) {
- frame = null;
- super.destroyTexImage(gl, imgTex);
+ protected final void destroyTexFrame(GL gl, TextureSequence.TextureFrame frame) {
+ super.destroyTexFrame(gl, frame);
}
private void validatePos() {
diff --git a/src/jogl/classes/jogamp/opengl/util/av/SyncedRingbuffer.java b/src/jogl/classes/jogamp/opengl/util/av/SyncedRingbuffer.java
index 5f5d69cf8..ea67387a0 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/SyncedRingbuffer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/SyncedRingbuffer.java
@@ -74,6 +74,8 @@ public class SyncedRingbuffer {
}
}
+ public final T[] getArray() { return array; }
+
public final int capacity() {
return capacity;
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
index 852e5149c..3680da1a8 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
@@ -116,25 +116,32 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
private static long[] symbolAddr;
private static final boolean ready;
+ private static final boolean libsLoaded;
static {
// native ffmpeg media player implementation is included in jogl_desktop and jogl_mobile
GLProfile.initSingleton();
boolean _ready = false;
+ boolean[] _libsLoaded= { false };
try {
- _ready = initSymbols();
+ _ready = initSymbols(_libsLoaded);
} catch (Throwable t) {
t.printStackTrace();
}
+ libsLoaded = _libsLoaded[0];
ready = _ready;
- if(!ready) {
- System.err.println("FFMPEG: Not Available");
+ if(!libsLoaded) {
+ System.err.println("LIB_AV Not Available");
+ } else if(!ready) {
+ System.err.println("LIB_AV Not Matching");
}
}
+ static boolean libsLoaded() { return libsLoaded; }
static boolean initSingleton() { return ready; }
- private static final boolean initSymbols() {
+ private static final boolean initSymbols(boolean[] libsLoaded) {
+ libsLoaded[0] = false;
final DynamicLibraryBundle dl = AccessController.doPrivileged(new PrivilegedAction() {
public DynamicLibraryBundle run() {
return new DynamicLibraryBundle(new FFMPEGDynamicLibraryBundleInfo());
@@ -148,6 +155,7 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
if(!dl.isToolLibComplete()) {
throw new RuntimeException("FFMPEG Tool libraries incomplete");
}
+ libsLoaded[0] = true;
if(symbolNames.length != symbolCount) {
throw new InternalError("XXX0 "+symbolNames.length+" != "+symbolCount);
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
index b6708b379..83a5960f1 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
@@ -44,11 +44,10 @@ import com.jogamp.opengl.util.GLPixelStorageModes;
import com.jogamp.opengl.util.av.AudioSink;
import com.jogamp.opengl.util.av.AudioSinkFactory;
import com.jogamp.opengl.util.texture.Texture;
-import com.jogamp.opengl.util.texture.TextureSequence;
+import com.jogamp.opengl.util.texture.TextureSequence.TextureFrame;
import jogamp.opengl.GLContextImpl;
-import jogamp.opengl.util.av.EGLMediaPlayerImpl;
-import jogamp.opengl.util.av.SyncedRingbuffer;
+import jogamp.opengl.util.av.GLMediaPlayerImpl;
/***
* Implementation utilizes Libav
@@ -102,7 +101,7 @@ import jogamp.opengl.util.av.SyncedRingbuffer;
*
*
*/
-public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
+public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
// Count of zeroed buffers to return before switching to real sample provider
private static final int TEMP_BUFFER_COUNT = 20;
@@ -114,21 +113,20 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
static final boolean available;
static {
- if(FFMPEGDynamicLibraryBundleInfo.initSingleton()) {
+ final boolean libAVGood = FFMPEGDynamicLibraryBundleInfo.initSingleton();
+ if( FFMPEGDynamicLibraryBundleInfo.libsLoaded() ) {
avUtilVersion = getAVVersion(getAvUtilVersion0());
avFormatVersion = getAVVersion(getAvFormatVersion0());
avCodecVersion = getAVVersion(getAvCodecVersion0());
System.err.println("LIB_AV Util : "+avUtilVersion);
System.err.println("LIB_AV Format: "+avFormatVersion);
System.err.println("LIB_AV Codec : "+avCodecVersion);
- initIDs0();
- available = true;
} else {
avUtilVersion = null;
avFormatVersion = null;
avCodecVersion = null;
- available = false;
}
+ available = libAVGood ? initIDs0() : false;
}
public static final boolean isAvailable() { return available; }
@@ -144,8 +142,6 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
//
protected long moviePtr = 0;
- protected long procAddrGLTexSubImage2D = 0;
- protected EGLMediaPlayerImpl.EGLTextureFrame lastTex = null;
protected GLPixelStorageModes psm;
protected PixelFormat vPixelFmt = null;
protected int vPlanes = 0;
@@ -161,21 +157,15 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
// Audio
//
- protected static final boolean USE_AUDIO_PUSHER = false;
protected final int AudioFrameCount = 8;
protected final AudioSink audioSink;
protected final int maxAvailableAudio;
protected AudioSink.AudioDataFormat chosenAudioFormat;
- protected final SyncedRingbuffer audioFramesBuffer =
- USE_AUDIO_PUSHER ? new SyncedRingbuffer(new AudioSink.AudioFrame[AudioFrameCount], false /* full */)
- : null;
public FFMPEGMediaPlayer() {
- super(TextureType.GL, false);
if(!available) {
throw new RuntimeException("FFMPEGMediaPlayer not available");
}
- setTextureCount(1);
moviePtr = createInstance0(DEBUG);
if(0==moviePtr) {
throw new GLException("Couldn't create FFMPEGInstance");
@@ -184,26 +174,15 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
audioSink = AudioSinkFactory.createDefault();
maxAvailableAudio = audioSink.getQueuedByteCount();
}
-
@Override
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- if(TextureType.GL == texType) {
- final Texture texture = super.createTexImageImpl(gl, idx, tex, texWidth, texHeight, true);
- lastTex = new EGLTextureFrame(null, texture, 0, 0);
- } else {
- throw new InternalError("n/a");
- }
- return lastTex;
+ protected final int validateTextureCount(int desiredTextureCount) {
+ return desiredTextureCount>1 ? desiredTextureCount : 2;
}
-
@Override
- protected void destroyTexImage(GL gl, TextureSequence.TextureFrame imgTex) {
- lastTex = null;
- super.destroyTexImage(gl, imgTex);
- }
-
+ protected final boolean requiresOffthreadGLCtx() { return true; }
+
@Override
- protected void destroyImpl(GL gl) {
+ protected final void destroyImpl(GL gl) {
if (moviePtr != 0) {
destroyInstance0(moviePtr);
moviePtr = 0;
@@ -211,15 +190,29 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected void initGLStreamImpl(GL gl, int[] texNames) throws IOException {
+ protected final void initGLStreamImpl(GL gl) throws IOException {
if(0==moviePtr) {
throw new GLException("FFMPEG native instance null");
}
+ {
+ final GLContextImpl ctx = (GLContextImpl)gl.getContext();
+ final ProcAddressTable pt = ctx.getGLProcAddressTable();
+ final long procAddrGLTexSubImage2D = getAddressFor(pt, "glTexSubImage2D");
+ if( 0 == procAddrGLTexSubImage2D ) {
+ throw new InternalError("glTexSubImage2D n/a in ProcAddressTable: "+pt.getClass().getName()+" of "+ctx.getGLVersion());
+ }
+ final long procAddrGLGetError = getAddressFor(pt, "glGetError");
+ if( 0 == procAddrGLGetError ) {
+ throw new InternalError("glGetError n/a in ProcAddressTable: "+pt.getClass().getName()+" of "+ctx.getGLVersion());
+ }
+ setGLFuncs0(moviePtr, procAddrGLTexSubImage2D, procAddrGLGetError);
+ }
+
final String urlS=urlConn.getURL().toExternalForm();
chosenAudioFormat = audioSink.initSink(audioSink.getPreferredFormat(), AudioFrameCount);
System.err.println("setURL: p1 "+this);
- setStream0(moviePtr, urlS, -1, -1, AudioFrameCount);
+ setStream0(moviePtr, urlS, -1, -1, AudioFrameCount); // issues updateAttributes*(..)
System.err.println("setURL: p2 "+this);
int tf, tif=GL.GL_RGBA; // texture format and internal format
@@ -239,12 +232,10 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
}
setTextureFormat(tif, tf);
setTextureType(GL.GL_UNSIGNED_BYTE);
- final GLContextImpl ctx = (GLContextImpl)gl.getContext();
- final ProcAddressTable pt = ctx.getGLProcAddressTable();
- procAddrGLTexSubImage2D = getAddressFor(pt, "glTexSubImage2D");
- if( 0 == procAddrGLTexSubImage2D ) {
- throw new InternalError("glTexSubImage2D n/a in ProcAddressTable: "+pt.getClass().getName()+" of "+ctx.getGLVersion());
- }
+ }
+ @Override
+ protected final TextureFrame createTexImage(GL gl, int texName) {
+ return new TextureFrame( createTexImageImpl(gl, texName, texWidth, texHeight, true) );
}
/**
@@ -263,107 +254,6 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
} ).longValue();
}
- private final void pushSound(ByteBuffer sampleData, int data_size, int audio_pts) {
- if( USE_AUDIO_PUSHER ) {
- if( audioPusher != null && audioPusher.isRunning() ) {
- try {
- audioFramesBuffer.putBlocking(new AudioSink.AudioFrame(sampleData, data_size, audio_pts));
- } catch (InterruptedException e) {
- e.printStackTrace(); // oops
- }
- }
- } else {
- pushAudioFrame(new AudioSink.AudioFrame(sampleData, data_size, audio_pts));
- }
- }
-
- private final void pushAudioFrame(AudioSink.AudioFrame audioFrame) {
- // poor mans audio sync ..
- final long now = System.currentTimeMillis();
- final long now_d = now - lastAudioTime;
- final long pts_d = audioFrame.audioPTS - lastAudioPTS;
- final long dt = (long) ( (float) ( pts_d - now_d ) / getPlaySpeed() ) ;
- final boolean sleep = dt > audio_dt_d;
- final long sleepP = dt - ( audio_dt_d / 2 );
- if(DEBUG) {
- final int qAT = audioSink.getQueuedTime();
- System.err.println("s: pts-a "+audioFrame.audioPTS+", qAT "+qAT+", pts-d "+pts_d+", now_d "+now_d+", dt "+dt+", sleep "+sleep+", sleepP "+sleepP+" ms");
- }
- if( sleep ) {
- try {
- Thread.sleep( sleepP );
- } catch (InterruptedException e) {
- e.printStackTrace(); // oops
- }
- lastAudioTime = System.currentTimeMillis();
- } else {
- lastAudioTime = now;
- }
- if( audioSink.isDataAvailable(audioFrame.dataSize) ) {
- audioSink.writeData(audioFrame);
- lastAudioPTS=audioFrame.audioPTS;
- }
- }
-
- class AudioPusher extends Thread {
- volatile boolean shallStop = false;
- volatile boolean isBlocked = false;
-
- AudioPusher() {
- setDaemon(true);
- }
- public void requestStop() {
- shallStop = true;
- if( isBlocked ) {
- interrupt();
- }
- }
- public boolean isRunning() { return !shallStop; }
-
- public void run() {
- setName(getName()+"-AudioPusher_"+AudioPusherInstanceId);
- AudioPusherInstanceId++;
-
- while( !shallStop ){
- final AudioSink.AudioFrame audioFrame;
- try {
- isBlocked = true;
- audioFrame = audioFramesBuffer.getBlocking(true /* clearRef */);
- } catch (InterruptedException e) {
- if( !shallStop ) {
- e.printStackTrace(); // oops
- }
- shallStop = true;
- return;
- }
- isBlocked = false;
-
- if( null != audioFrame ) {
- FFMPEGMediaPlayer.this.pushAudioFrame(audioFrame);
- }
- }
- }
- }
- static int AudioPusherInstanceId = 0;
- private AudioPusher audioPusher = null;
-
- private final void stopAudioPusher() {
- if( USE_AUDIO_PUSHER ) {
- if( null != audioPusher ) {
- audioPusher.requestStop();
- audioPusher = null;
- }
- audioFramesBuffer.clear(true);
- }
- }
- private final void startAudioPusher() {
- if( USE_AUDIO_PUSHER ) {
- stopAudioPusher();
- audioPusher = new AudioPusher();
- audioPusher.start();
- }
- }
-
private void updateAttributes2(int pixFmt, int planes, int bitsPerPixel, int bytesPerPixelPerPlane,
int lSz0, int lSz1, int lSz2,
int tWd0, int tWd1, int tWd2) {
@@ -413,7 +303,7 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
* Otherwise the call is delegated to it's super class.
*/
@Override
- public String getTextureLookupFunctionName(String desiredFuncName) throws IllegalStateException {
+ public final String getTextureLookupFunctionName(String desiredFuncName) throws IllegalStateException {
if(State.Uninitialized == state) {
throw new IllegalStateException("Instance not initialized: "+this);
}
@@ -434,7 +324,7 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
* e.g. YUV420P to RGB. Otherwise the call is delegated to it's super class.
*/
@Override
- public String getTextureLookupFragmentShaderImpl() throws IllegalStateException {
+ public final String getTextureLookupFragmentShaderImpl() throws IllegalStateException {
if(State.Uninitialized == state) {
throw new IllegalStateException("Instance not initialized: "+this);
}
@@ -465,134 +355,131 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected synchronized int getCurrentPositionImpl() {
+ protected final synchronized int getCurrentPositionImpl() {
return 0!=moviePtr ? getVideoPTS0(moviePtr) : 0;
}
@Override
- protected synchronized boolean setPlaySpeedImpl(float rate) {
+ public final int getAudioPTSImpl() { return 0; }
+
+ @Override
+ protected final synchronized boolean setPlaySpeedImpl(float rate) {
return true;
}
@Override
- public synchronized boolean startImpl() {
+ public final synchronized boolean startImpl() {
if(0==moviePtr) {
return false;
}
- startAudioPusher();
return true;
}
/** @return time position after issuing the command */
@Override
- public synchronized boolean pauseImpl() {
+ public final synchronized boolean pauseImpl() {
if(0==moviePtr) {
return false;
}
- stopAudioPusher();
return true;
}
/** @return time position after issuing the command */
@Override
- public synchronized boolean stopImpl() {
+ public final synchronized boolean stopImpl() {
if(0==moviePtr) {
return false;
}
- stopAudioPusher();
return true;
}
/** @return time position after issuing the command */
@Override
- protected synchronized int seekImpl(int msec) {
+ protected final synchronized int seekImpl(int msec) {
if(0==moviePtr) {
throw new GLException("FFMPEG native instance null");
}
- stopAudioPusher();
int pts0 = getVideoPTS0(moviePtr);
int pts1 = seek0(moviePtr, msec);
System.err.println("Seek: "+pts0+" -> "+msec+" : "+pts1);
- lastAudioPTS=pts1;
- lastVideoPTS=pts1;
- startAudioPusher();
return pts1;
}
@Override
- protected TextureSequence.TextureFrame getLastTextureImpl() {
- return lastTex;
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ if(0==moviePtr) {
+ throw new GLException("FFMPEG native instance null");
+ }
+ psm.setUnpackAlignment(gl, 1); // RGBA ? 4 : 1
+ int avPTS = 0;
+ try {
+ final Texture tex = nextFrame.getTexture();
+ gl.glActiveTexture(GL.GL_TEXTURE0+getTextureUnit());
+ tex.enable(gl);
+ tex.bind(gl);
+
+ /** Try decode up to 10 packets to find one containing video, i.e. vPTS > 0 */
+ for(int retry=10; 0 >= avPTS && 0 < retry; retry--) {
+ avPTS = readNextPacket0(moviePtr, textureTarget, textureFormat, textureType);
+ retry--;
+ }
+ } finally {
+ psm.restore(gl);
+ }
+ if( 0 < avPTS ) {
+ vSTS = avPTS;
+ nextFrame.setPTS(avPTS);
+ return true;
+ } else {
+ return false;
+ }
}
+ private final void pushSound(ByteBuffer sampleData, int data_size, int audio_pts) {
+ aSTS = audio_pts;
+ final AudioSink.AudioFrame frame = new AudioSink.AudioFrame(sampleData, data_size, audio_pts);
+ if( audioSink.isDataAvailable(frame.dataSize) ) {
+ audioSink.writeData(frame);
+ }
+ }
+
+ /** last audio streaming TS */
+ private int aSTS = 0;
+ /** last video streaming TS */
+ private int vSTS = 0;
+
private long lastAudioTime = 0;
- private int lastAudioPTS = 0;
private static final int audio_dt_d = 400;
private long lastVideoTime = 0;
- private int lastVideoPTS = 0;
private static final int video_dt_d = 9;
@Override
- protected TextureSequence.TextureFrame getNextTextureImpl(GL gl, boolean blocking) {
- if(0==moviePtr) {
- throw new GLException("FFMPEG native instance null");
- }
- if(null != lastTex) {
- psm.setUnpackAlignment(gl, 1); // RGBA ? 4 : 1
- try {
- final Texture tex = lastTex.getTexture();
- gl.glActiveTexture(GL.GL_TEXTURE0+getTextureUnit());
- tex.enable(gl);
- tex.bind(gl);
-
- if( USE_AUDIO_PUSHER ) {
- try {
- audioFramesBuffer.waitForFreeSlots(2);
- } catch (InterruptedException e) {
- e.printStackTrace(); // oops
- }
- }
-
- /* try decode 10 packets to find one containing video
- (res == 2) */
- int res = 0;
- int retry = 10;
- while(res!=2 && retry >= 0) {
- res = readNextPacket0(moviePtr, procAddrGLTexSubImage2D, textureTarget, textureFormat, textureType);
- retry--;
- }
- } finally {
- psm.restore(gl);
- }
- final int pts = getVideoPTS0(moviePtr); // this frame
- if(blocking) {
- // poor mans video sync .. TODO: off thread 'readNextPackage0(..)' on shared GLContext and multi textures/unit!
- final long now = System.currentTimeMillis();
- // Try sync video to audio
- final long now_d = now - lastAudioTime;
- final long pts_d = pts - lastAudioPTS - 444; /* hack 444 == play video 444ms ahead of audio */
- final long dt = Math.min(47, (long) ( (float) ( pts_d - now_d ) / getPlaySpeed() ) ) ;
- //final long dt = (long) ( (float) ( pts_d - now_d ) / getPlaySpeed() ) ;
- final boolean sleep = dt>video_dt_d && dt<1000 && audioSink.getQueuedByteCount()video_dt_d && dt<1000 && audioSink.getQueuedByteCount() 0, invalid == 0
+ */
+ private native int readNextPacket0(long moviePtr, int texTarget, int texFmt, int texType);
private native int seek0(long moviePtr, int position);
@@ -720,6 +615,5 @@ public class FFMPEGMediaPlayer extends EGLMediaPlayerImpl {
return null;
}
}
-
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
index aef98fcde..a21bb40a8 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
@@ -33,7 +33,6 @@ import java.net.URL;
import javax.media.opengl.GL;
import javax.media.opengl.GLException;
-import javax.media.opengl.GLProfile;
import com.jogamp.opengl.util.texture.TextureSequence;
@@ -49,17 +48,17 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
static final boolean available;
static {
+ available = false;
+ /** FIXME!
// OMX binding is included in jogl_desktop and jogl_mobile
GLProfile.initSingleton();
- available = initIDs0();
+ available = initIDs0(); */
}
public static final boolean isAvailable() { return available; }
protected long moviePtr = 0;
- protected TextureSequence.TextureFrame lastTex = null;
-
public OMXGLMediaPlayer() {
super(TextureType.KHRImage, true);
if(!available) {
@@ -76,17 +75,15 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected TextureSequence.TextureFrame createTexImage(GL gl, int idx, int[] tex) {
- final EGLTextureFrame eglTex = (EGLTextureFrame) super.createTexImage(gl, idx, tex);
- _setStreamEGLImageTexture2D(moviePtr, idx, tex[idx], eglTex.getImage(), eglTex.getSync());
- lastTex = eglTex;
+ protected TextureSequence.TextureFrame createTexImage(GL gl, int texName) {
+ final EGLTextureFrame eglTex = (EGLTextureFrame) super.createTexImage(gl, texName);
+ _setStreamEGLImageTexture2D(moviePtr, texName, eglTex.getImage(), eglTex.getSync());
return eglTex;
}
@Override
- protected void destroyTexImage(GL gl, TextureSequence.TextureFrame imgTex) {
- lastTex = null;
- super.destroyTexImage(gl, imgTex);
+ protected void destroyTexFrame(GL gl, TextureSequence.TextureFrame imgTex) {
+ super.destroyTexFrame(gl, imgTex);
}
@Override
@@ -99,7 +96,7 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected void initGLStreamImpl(GL gl, int[] texNames) throws IOException {
+ protected void initGLStreamImpl(GL gl) throws IOException {
if(0==moviePtr) {
throw new GLException("OMX native instance null");
}
@@ -119,6 +116,10 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
protected int getCurrentPositionImpl() {
return 0!=moviePtr ? _getCurrentPosition(moviePtr) : 0;
}
+ @Override
+ protected int getAudioPTSImpl() {
+ return getCurrentPositionImpl();
+ }
@Override
protected boolean setPlaySpeedImpl(float rate) {
@@ -168,24 +169,23 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected TextureSequence.TextureFrame getLastTextureImpl() {
- return lastTex;
- }
-
- @Override
- protected TextureSequence.TextureFrame getNextTextureImpl(GL gl, boolean blocking) {
+ protected boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
if(0==moviePtr) {
throw new GLException("OMX native instance null");
}
final int nextTex = _getNextTextureID(moviePtr, blocking);
if(0 < nextTex) {
- final TextureSequence.TextureFrame eglImgTex = texFrameMap.get(new Integer(_getNextTextureID(moviePtr, blocking)));
+ /* FIXME
+ final TextureSequence.TextureFrame eglImgTex =
+ texFrameMap.get(new Integer(_getNextTextureID(moviePtr, blocking)));
if(null!=eglImgTex) {
lastTex = eglImgTex;
- }
+ } */
}
- return lastTex;
+ return true;
}
+ @Override
+ protected void syncFrame2Audio(TextureFrame frame) { }
private String replaceAll(String orig, String search, String repl) {
String dest=null;
@@ -216,7 +216,7 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
private native void _setStream(long moviePtr, int textureNum, String path);
private native void _activateStream(long moviePtr);
- private native void _setStreamEGLImageTexture2D(long moviePtr, int i, int tex, long image, long sync);
+ private native void _setStreamEGLImageTexture2D(long moviePtr, int tex, long image, long sync);
private native int _seek(long moviePtr, int position);
private native void _setPlaySpeed(long moviePtr, float rate);
private native void _play(long moviePtr);
diff --git a/src/jogl/native/libav/ffmpeg_tool.h b/src/jogl/native/libav/ffmpeg_tool.h
index 5560b8617..2dff1110c 100644
--- a/src/jogl/native/libav/ffmpeg_tool.h
+++ b/src/jogl/native/libav/ffmpeg_tool.h
@@ -48,6 +48,11 @@
#include
#include
+#include
+
+typedef void (APIENTRYP PFNGLTEXSUBIMAGE2DPROC) (GLenum target, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLenum type, const GLvoid *pixels);
+typedef GLenum (APIENTRYP PFNGLGETERRORPROC) (void);
+
/**
* AV_TIME_BASE 1000000
*/
@@ -63,6 +68,9 @@ static inline int32_t my_av_q2i32(int32_t snum, AVRational a){
typedef struct {
int32_t verbose;
+ PFNGLTEXSUBIMAGE2DPROC procAddrGLTexSubImage2D;
+ PFNGLGETERRORPROC procAddrGLGetError;
+
AVFormatContext* pFormatCtx;
int32_t vid;
AVStream* pVStream;
diff --git a/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c b/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
index 623bdcac7..346ba6c07 100644
--- a/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
+++ b/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
@@ -33,8 +33,6 @@
#include
#include
-typedef void (APIENTRYP PFNGLTEXSUBIMAGE2DPROC) (GLenum target, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLenum type, const GLvoid *pixels);
-
static const char * const ClazzNameFFMPEGMediaPlayer = "jogamp/opengl/util/av/impl/FFMPEGMediaPlayer";
static jclass ffmpegMediaPlayerClazz = NULL;
@@ -583,22 +581,29 @@ JNIEXPORT void JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_setStre
_updateJavaAttributes(env, instance, pAV);
}
+JNIEXPORT void JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_setGLFuncs0
+ (JNIEnv *env, jobject instance, jlong ptr, jlong jProcAddrGLTexSubImage2D, jlong jProcAddrGLGetError)
+{
+ FFMPEGToolBasicAV_t *pAV = (FFMPEGToolBasicAV_t *)((void *)((intptr_t)ptr));
+ pAV->procAddrGLTexSubImage2D = (PFNGLTEXSUBIMAGE2DPROC) (intptr_t)jProcAddrGLTexSubImage2D;
+ pAV->procAddrGLGetError = (PFNGLGETERRORPROC) (intptr_t)jProcAddrGLGetError;
+}
+
JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNextPacket0
- (JNIEnv *env, jobject instance, jlong ptr, jlong jProcAddrGLTexSubImage2D, jint texTarget, jint texFmt, jint texType)
+ (JNIEnv *env, jobject instance, jlong ptr, jint texTarget, jint texFmt, jint texType)
{
FFMPEGToolBasicAV_t *pAV = (FFMPEGToolBasicAV_t *)((void *)((intptr_t)ptr));
- PFNGLTEXSUBIMAGE2DPROC procAddrGLTexSubImage2D = (PFNGLTEXSUBIMAGE2DPROC) (intptr_t)jProcAddrGLTexSubImage2D;
- jint res = 0; // 1 - audio, 2 - video
AVPacket packet;
int frameFinished;
+ jint resPTS = 0; // resulting current PTS: audio < 0, video > 0, invalid == 0
if(sp_av_read_frame(pAV->pFormatCtx, &packet)>=0) {
if(packet.stream_index==pAV->aid) {
// Decode audio frame
if(NULL == pAV->pAFrames) { // no audio registered
sp_av_free_packet(&packet);
- return res;
+ return 0;
}
AVFrame* pAFrameCurrent = pAV->pAFrames[pAV->aFrameCurrent];
pAV->aFrameCurrent = ( pAV->aFrameCurrent + 1 ) % pAV->aFrameCount ;
@@ -658,14 +663,13 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
jobject jSampleData = (*env)->NewDirectByteBuffer(env, pAFrameCurrent->data[0], data_size);
(*env)->CallVoidMethod(env, instance, jni_mid_pushSound, jSampleData, data_size, pAV->aPTS);
}
-
- res = 1;
+ resPTS = pAV->aPTS * -1; // Audio Frame!
}
} else if(packet.stream_index==pAV->vid) {
// Decode video frame
if(NULL == pAV->pVFrame) {
sp_av_free_packet(&packet);
- return res;
+ return 0;
}
int new_packet = 1;
@@ -696,7 +700,6 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
continue;
}
- res = 2;
// FIXME: Libav Binary compatibility! JAU01
const AVRational time_base = pAV->pVStream->time_base;
const int64_t pts = pAV->pVFrame->pkt_pts;
@@ -708,6 +711,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
pAV->vPTS, pAV->pVFrame->pkt_pts, time_base.num, time_base.den, (time_base.num/(double)time_base.den));
#endif
}
+ resPTS = pAV->vPTS; // Video Frame!
#if 0
printf("tex2D codec %dx%d - frame %dx%d - width %d tex / %d linesize, pixfmt 0x%X, texType 0x%x, texTarget 0x%x\n",
@@ -718,24 +722,51 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
// 1st plane or complete packed frame
// FIXME: Libav Binary compatibility! JAU01
- procAddrGLTexSubImage2D(texTarget, 0,
+ #if 0
+ GLenum glerr = pAV->procAddrGLGetError();
+ printf("TexSubImage2D.1 texTarget 0x%x, offset %d / %d, size %d x %d, fmt 0x%X, type 0x%X, pre-err 0x%X, ",
+ texTarget, 0, 0, pAV->vTexWidth[0], pAV->pVCodecCtx->height, texFmt, texType, glerr);
+ #endif
+ pAV->procAddrGLTexSubImage2D(texTarget, 0,
0, 0,
pAV->vTexWidth[0], pAV->pVCodecCtx->height,
texFmt, texType, pAV->pVFrame->data[0]);
+ #if 0
+ glerr = pAV->procAddrGLGetError();
+ printf("err 0x%X\n", glerr);
+ #endif
if(pAV->vPixFmt == PIX_FMT_YUV420P) {
// U plane
// FIXME: Libav Binary compatibility! JAU01
- procAddrGLTexSubImage2D(texTarget, 0,
+ #if 0
+ printf("TexSubImage2D.U texTarget 0x%x, offset %d / %d, size %d x %d, fmt 0x%X, type 0x%X, ",
+ texTarget, pAV->pVCodecCtx->width, 0, pAV->vTexWidth[1], pAV->pVCodecCtx->height/2,
+ texFmt, texType);
+ #endif
+ pAV->procAddrGLTexSubImage2D(texTarget, 0,
pAV->pVCodecCtx->width, 0,
pAV->vTexWidth[1], pAV->pVCodecCtx->height/2,
texFmt, texType, pAV->pVFrame->data[1]);
+ #if 0
+ glerr = pAV->procAddrGLGetError();
+ printf("err 0x%X\n", glerr);
+ #endif
// V plane
// FIXME: Libav Binary compatibility! JAU01
- procAddrGLTexSubImage2D(texTarget, 0,
+ #if 0
+ printf("TexSubImage2D.V texTarget 0x%x, offset %d / %d, size %d x %d, fmt 0x%X, type 0x%X, ",
+ texTarget, pAV->pVCodecCtx->width, pAV->pVCodecCtx->height/2, pAV->vTexWidth[2], pAV->pVCodecCtx->height/2,
+ texFmt, texType);
+ #endif
+ pAV->procAddrGLTexSubImage2D(texTarget, 0,
pAV->pVCodecCtx->width, pAV->pVCodecCtx->height/2,
pAV->vTexWidth[2], pAV->pVCodecCtx->height/2,
texFmt, texType, pAV->pVFrame->data[2]);
+ #if 0
+ glerr = pAV->procAddrGLGetError();
+ printf("err 0x%X\n", glerr);
+ #endif
} // FIXME: Add more planar formats !
}
}
@@ -745,7 +776,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
// TODO: check what release the packets memory.
// sp_av_free_packet(&packet);
}
- return res;
+ return resPTS;
}
JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_seek0
diff --git a/src/jogl/native/openmax/omx_tool.c b/src/jogl/native/openmax/omx_tool.c
index 16c43f6a8..26a3b93f1 100644
--- a/src/jogl/native/openmax/omx_tool.c
+++ b/src/jogl/native/openmax/omx_tool.c
@@ -1049,17 +1049,13 @@ void OMXToolBasicAV_SetStream(OMXToolBasicAV_t * pOMXAV, int vBufferNum, const K
DBG_PRINT( "SetStream X\n");
}
-void OMXToolBasicAV_SetStreamEGLImageTexture2D(OMXToolBasicAV_t * pOMXAV, KDint i, GLuint tex, EGLImageKHR image, EGLSyncKHR sync)
+void OMXToolBasicAV_SetStreamEGLImageTexture2D(OMXToolBasicAV_t * pOMXAV, GLuint tex, EGLImageKHR image, EGLSyncKHR sync)
{
if(NULL==pOMXAV) {
JoglCommon_throwNewRuntimeException(0, "OMX instance null\n");
return;
}
- DBG_PRINT( "SetStreamEGLImg %p #%d/%d t:%d i:%p s:%p..\n", pOMXAV, i, pOMXAV->vBufferNum, tex, image, sync);
- if(i<0||i>=pOMXAV->vBufferNum) {
- JoglCommon_throwNewRuntimeException(0, "Buffer index out of range: %d\n", i);
- return;
- }
+ DBG_PRINT( "SetStreamEGLImg %p count %d t:%d i:%p s:%p..\n", pOMXAV, pOMXAV->vBufferNum, tex, image, sync);
kdThreadMutexLock(pOMXAV->mutex);
{
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
index 20c28c3ea..adccecba0 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
@@ -136,7 +136,7 @@ public class TextureSequenceCubeES2 implements GLEventListener {
}
}
public void mouseWheelMoved(MouseEvent e) {
- System.err.println("XXX "+e);
+ // System.err.println("XXX "+e);
if( !e.isShiftDown() ) {
final float o = zoom;
final float d = e.getRotation()[1]/10f; // vertical: wheel
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
index 921710fed..3f979e16f 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
@@ -61,9 +61,11 @@ import com.jogamp.opengl.util.Animator;
import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.av.GLMediaPlayer.GLMediaEventListener;
import com.jogamp.opengl.util.av.GLMediaPlayerFactory;
+import com.jogamp.opengl.util.texture.TextureSequence.TextureFrame;
public class MovieCube implements GLEventListener, GLMediaEventListener {
static boolean waitForKey = false;
+ int textureCount = 3; // default - threaded
final URLConnection stream;
final float zoom0, rotx, roty;
TextureSequenceCubeES2 cube=null;
@@ -81,6 +83,10 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
this.roty = roty;
}
+ public void setTextureCount(int v) {
+ textureCount = v;
+ }
+
private final KeyListener keyAction = new KeyAdapter() {
public void keyReleased(KeyEvent e) {
if( !e.isPrintableKey() || e.isAutoRepeat() ) {
@@ -130,7 +136,7 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
}
@Override
- public void newFrameAvailable(GLMediaPlayer mp, long when) {
+ public void newFrameAvailable(GLMediaPlayer mp, TextureFrame newFrame, long when) {
// System.out.println("newFrameAvailable: "+mp+", when "+when);
}
@@ -151,7 +157,7 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
}
try {
System.out.println("p0 "+mPlayer);
- mPlayer.initGLStream(gl, stream);
+ mPlayer.initGLStream(gl, textureCount, stream);
System.out.println("p1 "+mPlayer);
} catch (Exception e) {
e.printStackTrace();
@@ -202,7 +208,7 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
public static void main(String[] args) throws MalformedURLException, IOException, InterruptedException {
int width = 510;
int height = 300;
- System.err.println("TexCubeES2.run()");
+ int textureCount = 3; // default - threaded
boolean forceES2 = false;
boolean forceES3 = false;
@@ -217,6 +223,9 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
} else if(args[i].equals("-height")) {
i++;
height = MiscUtils.atoi(args[i], height);
+ } else if(args[i].equals("-textureCount")) {
+ i++;
+ textureCount = MiscUtils.atoi(args[i], textureCount);
} else if(args[i].equals("-url")) {
i++;
url_s = args[i];
@@ -232,6 +241,7 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
waitForKey = true;
}
}
+ System.err.println("textureCount "+textureCount);
System.err.println("forceES2 "+forceES2);
System.err.println("forceES3 "+forceES3);
System.err.println("forceGL3 "+forceGL3);
@@ -264,7 +274,7 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
anim.stop();
}
});
- // anim.setUpdateFPSFrames(60, System.err);
+ anim.setUpdateFPSFrames(60, System.err);
anim.start();
window.setVisible(true);
}
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
index 7e0dcd909..90c73661a 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
@@ -66,9 +66,11 @@ import com.jogamp.opengl.util.glsl.ShaderState;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureCoords;
import com.jogamp.opengl.util.texture.TextureSequence;
+import com.jogamp.opengl.util.texture.TextureSequence.TextureFrame;
public class MovieSimple implements GLEventListener, GLMediaEventListener {
private int winWidth, winHeight;
+ int textureCount = 3; // default - threaded
private int prevMouseX; // , prevMouseY;
private int rotate = 0;
private boolean orthoProjection = true;
@@ -169,6 +171,9 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
public GLMediaPlayer getGLMediaPlayer() { return mPlayer; }
+ public void setTextureCount(int v) {
+ textureCount = v;
+ }
public void setScaleOrig(boolean v) {
mPlayerScaleOrig = v;
}
@@ -179,7 +184,7 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
}
@Override
- public void newFrameAvailable(GLMediaPlayer mp, long when) {
+ public void newFrameAvailable(GLMediaPlayer mp, TextureFrame newFrame, long when) {
// System.out.println("newFrameAvailable: "+mp+", when "+when);
}
@@ -252,7 +257,7 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
try {
System.out.println("p0 "+mPlayer+", shared "+mPlayerShared);
if(!mPlayerShared) {
- mPlayer.initGLStream(gl, stream);
+ mPlayer.initGLStream(gl, textureCount, stream);
}
tex = mPlayer.getLastTexture().getTexture();
System.out.println("p1 "+mPlayer+", shared "+mPlayerShared);
@@ -263,7 +268,8 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
if(!mPlayerShared) {
mPlayer.setTextureMinMagFilter( new int[] { GL.GL_NEAREST, GL.GL_LINEAR } );
}
- } catch (Exception glex) {
+ } catch (Exception glex) {
+ glex.printStackTrace();
if(!mPlayerShared && null != mPlayer) {
mPlayer.destroy(gl);
mPlayer = null;
@@ -506,6 +512,7 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
public static void main(String[] args) throws IOException, MalformedURLException {
int width = 640;
int height = 600;
+ int textureCount = 3; // default - threaded
boolean ortho = true;
boolean zoom = false;
@@ -522,6 +529,9 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
} else if(args[i].equals("-height")) {
i++;
height = MiscUtils.atoi(args[i], height);
+ } else if(args[i].equals("-textureCount")) {
+ i++;
+ textureCount = MiscUtils.atoi(args[i], textureCount);
} else if(args[i].equals("-es2")) {
forceES2 = true;
} else if(args[i].equals("-es3")) {
@@ -539,12 +549,14 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
url_s = args[i];
}
}
+ System.err.println("textureCount "+textureCount);
System.err.println("forceES2 "+forceES2);
System.err.println("forceES3 "+forceES3);
System.err.println("forceGL3 "+forceGL3);
System.err.println("forceGLDef "+forceGLDef);
final MovieSimple ms = new MovieSimple(new URL(url_s).openConnection());
+ ms.setTextureCount(textureCount);
ms.setScaleOrig(!zoom);
ms.setOrthoProjection(ortho);
@@ -570,6 +582,7 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
window.setSize(width, height);
window.setVisible(true);
final Animator anim = new Animator(window);
+ anim.setUpdateFPSFrames(60, System.err);
anim.start();
window.addWindowListener(new WindowAdapter() {
public void windowDestroyed(WindowEvent e) {
--
cgit v1.2.3
From c37629ea8fdcb11f7f8a18e37a4cde57d4ba6a01 Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Wed, 14 Aug 2013 07:02:59 +0200
Subject: GLMediaPlayer Multithreaded Decoding: GLMediaPlayer* (Part-3) - WIP
- GLMediaPlayer
- Remove State.Stopped and method stop() - redundant, use pause() / destroy()
- Add notion of stream IDs
- Add API doc: State / Stream-ID incl. html-anchor
- Expose video/audio PTS, ..
- Expose optional AudioSink
- Min multithreaded textureCount is 4 (EGL* and FFMPEG*)
- GLMediaPlayerImpl
- Move AudioSink rel. impl. to this class,
allowing a tight video implementation reusing logic.
- Remove 'synchronized' methods, synchronize on State
where applicable
- implement new methods (see above)
- playSpeed is handled partially in AudioSink.
If it exeeds AudioSink's capabilities, drop audio and rely solely on video sync.
- video sync (WIP)
- video pts delay based on geometric weight
- reset video SCR if 'out of range', resync w/ PTS
-
- FramePusher
- allow interruption when pausing/stopping,
while waiting for next avail free frame to decode.
- FFMPEGMediaPlayer
- Add proper AudioDataFormat negotiation AudioSink <-> libav
- Parse libav's SampleFormat
- Remove AudioSink interaction (moved to GLMediaPlayerImpl)
- Tests (MovieSimple, MovieCube):
- Add aid/vid selection
- Add KeyListener for actions: seek(..), play()/pause(), setPlaySpeed(..)
- Dump perf-string each 2s
- TODO:
- Add audio sync in AudioSink, similar to GLMediaPlayer's weighted video delay,
here: drop audio frames.
---
make/scripts/tests.sh | 6 +-
.../com/jogamp/opengl/util/av/GLMediaPlayer.java | 146 ++--
.../opengl/util/av/GLMediaPlayerFactory.java | 5 +-
.../opengl/util/texture/TextureSequence.java | 5 +-
.../android/av/AndroidGLMediaPlayerAPI14.java | 53 +-
.../jogamp/opengl/util/av/EGLMediaPlayerImpl.java | 2 +-
.../jogamp/opengl/util/av/GLMediaPlayerImpl.java | 740 +++++++++++++++------
.../jogamp/opengl/util/av/NullGLMediaPlayer.java | 27 +-
.../opengl/util/av/impl/FFMPEGMediaPlayer.java | 232 ++++---
.../opengl/util/av/impl/OMXGLMediaPlayer.java | 25 +-
src/jogl/native/libav/ffmpeg_tool.h | 19 +-
.../jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c | 223 ++++---
.../opengl/test/android/MovieCubeActivity0.java | 3 +-
.../opengl/test/android/MovieSimpleActivity0.java | 3 +-
.../opengl/test/android/MovieSimpleActivity1.java | 4 +-
.../jogl/demos/es2/TextureSequenceCubeES2.java | 23 +-
.../test/junit/jogl/demos/es2/av/MovieCube.java | 157 +++--
.../test/junit/jogl/demos/es2/av/MovieSimple.java | 228 +++++--
18 files changed, 1283 insertions(+), 618 deletions(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/make/scripts/tests.sh b/make/scripts/tests.sh
index ba4589d14..82e9fb92b 100644
--- a/make/scripts/tests.sh
+++ b/make/scripts/tests.sh
@@ -131,7 +131,9 @@ function jrun() {
#D_ARGS="-Djogl.1thread=true -Djogl.debug.Threading"
#D_ARGS="-Djogl.debug.DebugGL -Djogl.debug.TraceGL -Djogl.debug.GLContext.TraceSwitch -Djogl.debug=all"
#D_ARGS="-Djogamp.debug.IOUtil -Djogl.debug.GLSLCode -Djogl.debug.GLMediaPlayer"
+ #D_ARGS="-Djogl.debug.GLMediaPlayer"
#D_ARGS="-Djogl.debug.GLMediaPlayer -Djogl.debug.AudioSink"
+ #D_ARGS="-Djogl.debug.AudioSink"
#D_ARGS="-Djogl.debug.GLArrayData"
#D_ARGS="-Djogl.debug.GLDrawable"
#D_ARGS="-Djogl.debug.EGLDisplayUtil -Dnativewindow.debug.GraphicsConfiguration -Djogl.debug.GLDrawable"
@@ -314,8 +316,8 @@ function testawtswt() {
# av demos
#
#testnoawt jogamp.opengl.openal.av.ALDummyUsage $*
-testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
-#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
+#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
+testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
#
# core/newt (testnoawt and testawt)
diff --git a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
index a36bce305..fae88ea18 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
@@ -34,19 +34,37 @@ import javax.media.opengl.GL;
import javax.media.opengl.GLException;
import jogamp.opengl.Debug;
+import jogamp.opengl.util.av.GLMediaPlayerImpl;
import com.jogamp.opengl.util.texture.TextureSequence;
/**
- * Lifecycle of an GLMediaPlayer:
+ * GLMediaPlayer interface specifies a {@link TextureSequence}
+ * with a video stream as it's source.
+ *
+ * Audio maybe supported and played back internally or via an {@link AudioSink} implementation,
+ * if an audio stream is selected in {@link #initGLStream(GL, int, URLConnection, int, int)}.
+ *
* Current implementations (check each API doc link for details):
*
@@ -76,14 +94,21 @@ import com.jogamp.opengl.util.texture.TextureSequence;
*/
public interface GLMediaPlayer extends TextureSequence {
public static final boolean DEBUG = Debug.debug("GLMediaPlayer");
+
+ /** Constant {@value} for mute or not available. See Audio and video Stream IDs. */
+ public static final int STREAM_ID_NONE = -2;
+ /** Constant {@value} for auto or unspecified. See Audio and video Stream IDs. */
+ public static final int STREAM_ID_AUTO = -1;
public interface GLMediaEventListener extends TexSeqEventListener {
- static final int EVENT_CHANGE_SIZE = 1<<0;
- static final int EVENT_CHANGE_FPS = 1<<1;
- static final int EVENT_CHANGE_BPS = 1<<2;
- static final int EVENT_CHANGE_LENGTH = 1<<3;
- static final int EVENT_CHANGE_CODEC = 1<<3;
+ static final int EVENT_CHANGE_VID = 1<<0;
+ static final int EVENT_CHANGE_AID = 1<<1;
+ static final int EVENT_CHANGE_SIZE = 1<<2;
+ static final int EVENT_CHANGE_FPS = 1<<3;
+ static final int EVENT_CHANGE_BPS = 1<<4;
+ static final int EVENT_CHANGE_LENGTH = 1<<5;
+ static final int EVENT_CHANGE_CODEC = 1<<6;
/**
* @param mp the event source
@@ -93,8 +118,11 @@ public interface GLMediaPlayer extends TextureSequence {
public void attributesChanges(GLMediaPlayer mp, int event_mask, long when);
}
+ /**
+ * See GLMediaPlayer Lifecycle.
+ */
public enum State {
- Uninitialized(0), Stopped(1), Playing(2), Paused(3);
+ Uninitialized(0), Playing(1), Paused(2);
public final int id;
@@ -120,74 +148,106 @@ public interface GLMediaPlayer extends TextureSequence {
* Sets the stream to be used. Initializes all stream related states inclusive OpenGL ones,
* if gl is not null.
*
* @param gl current GL object. If null, no video output and textures will be available.
* @param textureCount desired number of buffered textures to be decoded off-thread, use 1 for on-thread decoding.
* @param urlConn the stream connection
+ * @param vid video stream id, see audio and video Stream IDs
+ * @param aid video stream id, see audio and video Stream IDs
* @return the new state
*
* @throws IllegalStateException if not invoked in state Uninitialized
* @throws IOException in case of difficulties to open or process the stream
* @throws GLException in case of difficulties to initialize the GL resources
*/
- public State initGLStream(GL gl, int textureCount, URLConnection urlConn) throws IllegalStateException, GLException, IOException;
+ public State initGLStream(GL gl, int textureCount, URLConnection urlConn, int vid, int aid) throws IllegalStateException, GLException, IOException;
+
+ /**
+ * If implementation uses a {@link AudioSink}, it's instance will be returned.
+ *
+ * The {@link AudioSink} instance is available after {@link #initGLStream(GL, int, URLConnection, int, int)},
+ * if used by implementation.
+ *
+ */
+ public AudioSink getAudioSink();
/**
* Releases the GL and stream resources.
*
*/
public State destroy(GL gl);
- public void setPlaySpeed(float rate);
+ /**
+ * Sets the playback speed.
+ *
+ * Play speed is set to normal, i.e. 1.0f
+ * if abs(1.0f - rate) < 0.01f to simplify test.
+ *
+ * @return true if successful, otherwise false, i.e. due to unsupported value range of implementation.
+ */
+ public boolean setPlaySpeed(float rate);
public float getPlaySpeed();
/**
- * Stopped/Paused -> Playing
+ * GLMediaPlayer Lifecycle: Paused -> Playing
*/
- public State start();
+ public State play();
/**
- * Playing -> Paused
+ * GLMediaPlayer Lifecycle: Playing -> Paused
*/
public State pause();
/**
- * Playing/Paused -> Stopped
+ * Allowed in state Playing and Paused, otherwise ignored,
+ * see GLMediaPlayer Lifecycle.
+ *
+ * @param msec absolute desired time position in milliseconds
+ * @return time current position in milliseconds, after seeking to the desired position
+ **/
+ public int seek(int msec);
+
+ /**
+ * See GLMediaPlayer Lifecycle.
+ * @return the current state, either Uninitialized, Playing, Paused
*/
- public State stop();
+ public State getState();
/**
- * @return the current state, either Uninitialized, Stopped, Playing, Paused
+ * Return the video stream id, see audio and video Stream IDs.
*/
- public State getState();
+ public int getVID();
/**
- * @return current streaming position in milliseconds
- **/
- public int getCurrentPosition();
-
+ * Return the audio stream id, see audio and video Stream IDs.
+ */
+ public int getAID();
+
+ /**
+ * @return the current decoded frame count since {@link #initGLStream(GL, int, URLConnection, int, int)}.
+ */
+ public int getDecodedFrameCount();
+
+ /**
+ * @return the current presented frame count since {@link #initGLStream(GL, int, URLConnection, int, int)},
+ * increased by {@link #getNextTexture(GL, boolean)}.
+ */
+ public int getPresentedFrameCount();
+
/**
- * @return current video PTS in milliseconds of {@link #getLastTexture()}
+ * @return current video presentation timestamp (PTS) in milliseconds of {@link #getLastTexture()}
**/
public int getVideoPTS();
/**
- * @return current audio PTS in milliseconds.
+ * @return current audio presentation timestamp (PTS) in milliseconds.
**/
public int getAudioPTS();
- /**
- * Allowed in state Stopped, Playing and Paused, otherwise ignored.
- *
- * @param msec absolute desired time position in milliseconds
- * @return time current position in milliseconds, after seeking to the desired position
- **/
- public int seek(int msec);
-
/**
* {@inheritDoc}
*/
@@ -225,7 +285,13 @@ public interface GLMediaPlayer extends TextureSequence {
* Warning: Optional information, may not be supported by implementation.
* @return the total number of video frames
*/
- public long getTotalFrames();
+ public int getVideoFrames();
+
+ /**
+ * Warning: Optional information, may not be supported by implementation.
+ * @return the total number of audio frames
+ */
+ public int getAudioFrames();
/**
* @return total duration of stream in msec.
@@ -262,6 +328,8 @@ public interface GLMediaPlayer extends TextureSequence {
public String toString();
+ public String getPerfString();
+
public void addEventListener(GLMediaEventListener l);
public void removeEventListener(GLMediaEventListener l);
diff --git a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
index 2707dd6d2..c7e1ab5e6 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
@@ -47,10 +47,13 @@ public class GLMediaPlayerFactory {
sink = create(cl, FFMPEGMediaPlayerClazzName);
}
if( null == sink ) {
- sink = new NullGLMediaPlayer();
+ sink = createNull();
}
return sink;
}
+ public static GLMediaPlayer createNull() {
+ return new NullGLMediaPlayer();
+ }
public static GLMediaPlayer create(final ClassLoader cl, String implName) {
try {
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 3f739b2cc..50801e791 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -110,9 +110,12 @@ public interface TextureSequence {
* to associated related data.
*/
public static class TextureFrame {
+ /** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE {@value}. */
+ public static final int INVALID_PTS = Integer.MIN_VALUE;
+
public TextureFrame(Texture t) {
texture = t;
- pts = 0;
+ pts = INVALID_PTS;
}
public final Texture getTexture() { return texture; }
diff --git a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
index 765cda084..e14642c34 100644
--- a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
+++ b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
@@ -28,13 +28,13 @@
package jogamp.opengl.android.av;
import java.io.IOException;
-import java.nio.Buffer;
import javax.media.opengl.GL;
import javax.media.opengl.GLES2;
import com.jogamp.common.os.AndroidVersion;
import com.jogamp.common.os.Platform;
+import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureSequence;
@@ -100,7 +100,7 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected final boolean startImpl() {
+ protected final boolean playImpl() {
if(null != mp) {
try {
mp.start();
@@ -130,22 +130,6 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
return false;
}
- @Override
- protected final boolean stopImpl() {
- if(null != mp) {
- wakeUp(false);
- try {
- mp.stop();
- return true;
- } catch (IllegalStateException ise) {
- if(DEBUG) {
- ise.printStackTrace();
- }
- }
- }
- return false;
- }
-
@Override
protected final int seekImpl(int msec) {
if(null != mp) {
@@ -165,15 +149,19 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected final int getCurrentPositionImpl() { return null != mp ? mp.getCurrentPosition() : 0; }
-
- @Override
- protected final int getAudioPTSImpl() { return getCurrentPositionImpl(); }
+ protected final int getAudioPTSImpl() { return null != mp ? mp.getCurrentPosition() : 0; }
@Override
protected final void destroyImpl(GL gl) {
if(null != mp) {
wakeUp(false);
+ try {
+ mp.stop();
+ } catch (IllegalStateException ise) {
+ if(DEBUG) {
+ ise.printStackTrace();
+ }
+ }
mp.release();
mp = null;
}
@@ -198,8 +186,13 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected final void initGLStreamImpl(GL gl) throws IOException {
+ protected final void initGLStreamImpl(GL gl, int vid, int aid) throws IOException {
if(null!=mp && null!=urlConn) {
+ if( GLMediaPlayer.STREAM_ID_NONE == aid ) {
+ mp.setVolume(0f, 0f);
+ // FIXME: Disable audio handling
+ } // else FIXME: Select aid !
+ // Note: Both FIXMEs seem to be n/a via Android's MediaPlayer -> Switch to API level 16 MediaCodec/MediaExtractor ..
try {
final Uri uri = Uri.parse(urlConn.getURL().toExternalForm());
mp.setDataSource(StaticContext.getContext(), uri);
@@ -213,20 +206,18 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
if( null == stex ) {
throw new InternalError("XXX");
}
- final Surface surf = new Surface(stex);
- mp.setSurface(surf);
- surf.release();
mp.setSurface(null);
try {
mp.prepare();
} catch (IOException ioe) {
throw new IOException("MediaPlayer failed to process stream <"+urlConn.getURL().toExternalForm()+">: "+ioe.getMessage(), ioe);
}
+ final int r_aid = GLMediaPlayer.STREAM_ID_NONE == aid ? GLMediaPlayer.STREAM_ID_NONE : GLMediaPlayer.STREAM_ID_AUTO;
final String icodec = "android";
- updateAttributes(mp.getVideoWidth(), mp.getVideoHeight(),
- 0, 0, 0,
- 0f, 0, mp.getDuration(),
- icodec, icodec);
+ updateAttributes(GLMediaPlayer.STREAM_ID_AUTO, r_aid,
+ mp.getVideoWidth(), mp.getVideoHeight(), 0,
+ 0, 0, 0f,
+ 0, 0, mp.getDuration(), icodec, icodec);
}
}
@@ -264,8 +255,6 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
return true;
}
- @Override
- protected final void syncFrame2Audio(TextureFrame frame) {}
@Override
protected final TextureSequence.TextureFrame createTexImage(GL gl, int texName) {
diff --git a/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java b/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
index 57d5ff625..db2146cdc 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/EGLMediaPlayerImpl.java
@@ -84,7 +84,7 @@ public abstract class EGLMediaPlayerImpl extends GLMediaPlayerImpl {
}
@Override
protected final int validateTextureCount(int desiredTextureCount) {
- return desiredTextureCount>1 ? desiredTextureCount : 2;
+ return desiredTextureCount>2 ? Math.max(4, desiredTextureCount) : 2;
}
@Override
diff --git a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
index bc297dc21..c1cfc0d95 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
@@ -42,6 +42,7 @@ import javax.media.opengl.GLES2;
import javax.media.opengl.GLException;
import javax.media.opengl.GLProfile;
+import com.jogamp.opengl.util.av.AudioSink;
import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureSequence;
@@ -62,7 +63,11 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected static final String unknown = "unknown";
- protected State state;
+ /** Default texture count w/o threading, value {@value}. */
+ protected static final int TEXTURE_COUNT_DEFAULT = 2;
+
+ protected volatile State state;
+ private Object stateLock = new Object();
protected int textureCount;
protected int textureTarget;
@@ -79,30 +84,72 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected volatile float playSpeed = 1.0f;
- /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ protected int vid = GLMediaPlayer.STREAM_ID_AUTO;
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ protected int aid = GLMediaPlayer.STREAM_ID_AUTO;
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int width = 0;
- /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int height = 0;
- /** Video fps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Video fps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected float fps = 0;
- /** Stream bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ protected int frame_period = 0;
+ /** Stream bps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int bps_stream = 0;
- /** Video bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Video bps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int bps_video = 0;
- /** Audio bps. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Audio bps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int bps_audio = 0;
- /** In frames. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
- protected int totalFrames = 0;
- /** In ms. Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** In frames. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ protected int videoFrames = 0;
+ /** In frames. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ protected int audioFrames = 0;
+ /** In ms. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int duration = 0;
- /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected String acodec = unknown;
- /** Shall be set by the {@link #initGLStreamImpl(GL)} method implementation. */
+ /** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected String vcodec = unknown;
- protected int frameNumber = 0;
- protected int currentVideoPTS = 0;
+ protected volatile int decodedFrameCount = 0;
+ protected int presentedFrameCount = 0;
+ protected volatile int video_pts_last = 0;
+
+ /** See {@link #getAudioSink()}. Set by implementation if used from within {@link #initGLStreamImpl(GL, int, int)}! */
+ protected AudioSink audioSink = null;
+ protected boolean audioSinkPlaySpeedSet = false;
+
+ /** System Clock Reference (SCR) of first audio PTS at start time. */
+ private long audio_scr_t0 = 0;
+ private boolean audioSCR_reset = true;
+ /** System Clock Reference (SCR) of first video frame at start time. */
+ private long video_scr_t0 = 0;
+ /** System Clock Reference (SCR) PTS offset, i.e. first video PTS at start time. */
+ private int video_scr_pts = 0;
+ /** Cumulative video pts diff. */
+ private float video_dpts_cum = 0;
+ /** Cumulative video frames. */
+ private int video_dpts_count = 0;
+ /** Number of min frame count required for video cumulative sync. */
+ private static final int VIDEO_DPTS_NUM = 20;
+ /** Cumulative coefficient, value {@value}. */
+ private static final float VIDEO_DPTS_COEFF = 0.7943282f; // (float) Math.exp(Math.log(0.01) / VIDEO_DPTS_NUM);
+ /** Maximum valid video pts diff. */
+ private static final int VIDEO_DPTS_MAX = 5000; // 5s max diff
+ /** Trigger video PTS reset with given cause as bitfield. */
+ private volatile int videoSCR_reset = 0;
+
+ private final boolean isSCRCause(int bit) { return 0 != ( bit & videoSCR_reset); }
+ /** SCR reset due to: Start, Resume, Seek, .. */
+ private static final int SCR_RESET_FORCE = 1 << 0;
+ /** SCR reset due to: PlaySpeed */
+ private static final int SCR_RESET_SPEED = 1 << 1;
+
+ /** Latched video PTS reset, to wait until valid pts after invalidation of cached ones. Currently [1..{@link #VIDEO_DPTS_NUM}] frames. */
+ private int videoSCR_reset_latch = 0;
+
protected SyncedRingbuffer videoFramesFree = null;
protected SyncedRingbuffer videoFramesDecoded = null;
protected volatile TextureFrame lastFrame = null;
@@ -201,144 +248,205 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
@Override
- public final float getPlaySpeed() {
- return playSpeed;
- }
+ public final int getDecodedFrameCount() { return decodedFrameCount; }
+
+ @Override
+ public final int getPresentedFrameCount() { return this.presentedFrameCount; }
+
+ @Override
+ public final int getVideoPTS() { return video_pts_last; }
@Override
- public final synchronized void setPlaySpeed(float rate) {
- if(State.Uninitialized != state && setPlaySpeedImpl(rate)) {
- playSpeed = rate;
+ public final int getAudioPTS() {
+ if( State.Uninitialized != state ) {
+ return getAudioPTSImpl();
}
- if(DEBUG) { System.err.println("SetPlaySpeed: "+toString()); }
+ return 0;
}
- protected abstract boolean setPlaySpeedImpl(float rate);
-
- public final State start() {
- switch( state ) {
- case Stopped:
- /** fall-through intended */
- case Paused:
- if( startImpl() ) {
- resumeFramePusher();
- state = State.Playing;
- }
- default:
+ /** Override if not using audioSink! */
+ protected int getAudioPTSImpl() {
+ if( null != audioSink ) {
+ return audioSink.getPTS();
+ } else {
+ return 0;
}
- if(DEBUG) { System.err.println("Start: "+toString()); }
- return state;
}
- protected abstract boolean startImpl();
- public final State pause() {
- if( State.Playing == state && pauseImpl() ) {
- pauseFramePusher();
- state = State.Paused;
+ public final State getState() { return state; }
+
+ public final State play() {
+ synchronized( stateLock ) {
+ switch( state ) {
+ case Paused:
+ if( playImpl() ) {
+ resetAudioVideoSCR(SCR_RESET_FORCE);
+ resumeFramePusher();
+ if( null != audioSink ) {
+ audioSink.play();
+ }
+ state = State.Playing;
+ }
+ default:
+ }
+ if(DEBUG) { System.err.println("Start: "+toString()); }
+ return state;
}
- if(DEBUG) { System.err.println("Pause: "+toString()); }
- return state;
}
- protected abstract boolean pauseImpl();
+ protected abstract boolean playImpl();
- public final State stop() {
- switch( state ) {
- case Playing:
- /** fall-through intended */
- case Paused:
- if( stopImpl() ) {
+ public final State pause() {
+ synchronized( stateLock ) {
+ if( State.Playing == state ) {
+ State _state = state;
+ state = State.Paused;
+ if( pauseImpl() ) {
+ _state = State.Paused;
pauseFramePusher();
- state = State.Stopped;
+ if( null != audioSink ) {
+ audioSink.pause();
+ }
}
- default:
+ state = _state;
+ }
+ if(DEBUG) { System.err.println("Pause: "+toString()); }
+ return state;
}
- if(DEBUG) { System.err.println("Stop: "+toString()); }
- return state;
}
- protected abstract boolean stopImpl();
+ protected abstract boolean pauseImpl();
- @Override
- public final int getCurrentPosition() {
- if( State.Uninitialized != state ) {
- return getCurrentPositionImpl();
+ public final int seek(int msec) {
+ synchronized( stateLock ) {
+ final int pts1;
+ switch(state) {
+ case Playing:
+ case Paused:
+ final State _state = state;
+ state = State.Paused;
+ pauseFramePusher();
+ resetAudioVideoSCR(SCR_RESET_FORCE);
+ pts1 = seekImpl(msec);
+ if( null != audioSink ) {
+ audioSink.flush();
+ if( State.Playing == _state ) {
+ audioSink.play(); // cont. w/ new data
+ }
+ }
+ resumeFramePusher();
+ state = _state;
+ break;
+ default:
+ pts1 = 0;
+ }
+ if(DEBUG) { System.err.println("Seek("+msec+"): "+toString()); }
+ return pts1;
}
- return 0;
}
- protected abstract int getCurrentPositionImpl();
+ protected abstract int seekImpl(int msec);
@Override
- public final int getVideoPTS() { return currentVideoPTS; }
+ public final float getPlaySpeed() {
+ return playSpeed;
+ }
@Override
- public final int getAudioPTS() {
- if( State.Uninitialized != state ) {
- return getAudioPTSImpl();
+ public final boolean setPlaySpeed(float rate) {
+ synchronized( stateLock ) {
+ boolean res = false;
+ if(State.Uninitialized != state ) {
+ if( rate > 0.01f ) {
+ if( Math.abs(1.0f - rate) < 0.01f ) {
+ rate = 1.0f;
+ }
+ if( setPlaySpeedImpl(rate) ) {
+ resetAudioVideoSCR(SCR_RESET_SPEED);
+ playSpeed = rate;
+ if(DEBUG) { System.err.println("SetPlaySpeed: "+toString()); }
+ res = true;
+ }
+ }
+ }
+ return res;
}
- return 0;
- }
- protected abstract int getAudioPTSImpl();
-
- public final int seek(int msec) {
- final int pts1;
- switch(state) {
- case Stopped:
- case Playing:
- case Paused:
- pauseFramePusher();
- pts1 = seekImpl(msec);
- currentVideoPTS=pts1;
- resumeFramePusher();
- break;
- default:
- pts1 = 0;
+ }
+ /**
+ * Override if not using AudioSink, or AudioSink's {@link AudioSink#setPlaySpeed(float)} is not sufficient!
+ *
+ * AudioSink shall respect !audioSinkPlaySpeedSet to determine data_size
+ * at {@link AudioSink#enqueueData(com.jogamp.opengl.util.av.AudioSink.AudioFrame)}.
+ *
+ */
+ protected boolean setPlaySpeedImpl(float rate) {
+ if( null != audioSink ) {
+ audioSinkPlaySpeedSet = audioSink.setPlaySpeed(rate);
}
- if(DEBUG) { System.err.println("Seek("+msec+"): "+toString()); }
- return pts1;
+ // still true, even if audioSink rejects command since we deal w/ video sync
+ // and AudioSink w/ audioSinkPlaySpeedSet at enqueueData(..).
+ return true;
}
- protected abstract int seekImpl(int msec);
-
- public final State getState() { return state; }
-
+
@Override
- public final State initGLStream(GL gl, int reqTextureCount, URLConnection urlConn) throws IllegalStateException, GLException, IOException {
- if(State.Uninitialized != state) {
- throw new IllegalStateException("Instance not in state "+State.Uninitialized+", but "+state+", "+this);
- }
- this.urlConn = urlConn;
- if (this.urlConn != null) {
- try {
- if( null != gl ) {
- removeAllTextureFrames(gl);
- textureCount = validateTextureCount(reqTextureCount);
- if( textureCount < 2 ) {
- throw new InternalError("Validated texture count < 2: "+textureCount);
- }
- initGLStreamImpl(gl); // also initializes width, height, .. etc
- videoFramesFree = new SyncedRingbuffer(createTexFrames(gl, textureCount), true /* full */);
- if( 2 < textureCount ) {
- videoFramesDecoded = new SyncedRingbuffer(new TextureFrame[textureCount], false /* full */);
- framePusher = new FramePusher(gl, requiresOffthreadGLCtx());
- framePusher.doStart();
- } else {
- videoFramesDecoded = null;
+ public final State initGLStream(GL gl, int reqTextureCount, URLConnection urlConn, int vid, int aid) throws IllegalStateException, GLException, IOException {
+ synchronized( stateLock ) {
+ if(State.Uninitialized != state) {
+ throw new IllegalStateException("Instance not in state "+State.Uninitialized+", but "+state+", "+this);
+ }
+ decodedFrameCount = 0;
+ presentedFrameCount = 0;
+ this.urlConn = urlConn;
+ if (this.urlConn != null) {
+ try {
+ if( null != gl ) {
+ removeAllTextureFrames(gl);
+ textureCount = validateTextureCount(reqTextureCount);
+ if( textureCount < TEXTURE_COUNT_DEFAULT ) {
+ throw new InternalError("Validated texture count < "+TEXTURE_COUNT_DEFAULT+": "+textureCount);
+ }
+ initGLStreamImpl(gl, vid, aid); // also initializes width, height, .. etc
+ videoFramesFree = new SyncedRingbuffer(createTexFrames(gl, textureCount), true /* full */);
+ if( TEXTURE_COUNT_DEFAULT < textureCount ) {
+ videoFramesDecoded = new SyncedRingbuffer(new TextureFrame[textureCount], false /* full */);
+ framePusher = new FramePusher(gl, requiresOffthreadGLCtx());
+ framePusher.doStart();
+ } else {
+ videoFramesDecoded = null;
+ }
+ lastFrame = videoFramesFree.getBlocking(false /* clearRef */ );
+ state = State.Paused;
}
- lastFrame = videoFramesFree.getBlocking(false /* clearRef */ );
+ return state;
+ } catch (Throwable t) {
+ throw new GLException("Error initializing GL resources", t);
}
- state = State.Stopped;
- return state;
- } catch (Throwable t) {
- throw new GLException("Error initializing GL resources", t);
}
+ return state;
}
- return state;
}
+ /**
+ * Implementation shall set the following set of data here
+ * @see #vid
+ * @see #aid
+ * @see #width
+ * @see #height
+ * @see #fps
+ * @see #bps_stream
+ * @see #videoFrames
+ * @see #audioFrames
+ * @see #acodec
+ * @see #vcodec
+ */
+ protected abstract void initGLStreamImpl(GL gl, int vid, int aid) throws IOException;
+
/**
* Returns the validated number of textures to be handled.
*
- * Default is always 2 textures, last texture and the decoding texture.
+ * Default is 2 textures w/o threading, last texture and the decoding texture.
+ *
+ *
+ * > 2 textures is used for threaded decoding, a minimum of 4 textures seems reasonable in this case.
*
+ * Note: All {@link AudioSink} operations are performed from {@link GLMediaPlayerImpl},
+ * i.e. {@link #play()}, {@link #pause()}, {@link #seek(int)}, {@link #setPlaySpeed(float)}, {@link #getAudioPTS()}.
+ *
+ *
+ * Implementations using an {@link AudioSink} shall write it's instance to {@link #audioSink}
+ * from within their {@link #initGLStreamImpl(GL, int, int)} implementation.
+ *
+ * Recommendation of audio/video pts time lead/lag at production:
+ *
+ *
Overall: +40ms and -60ms audio ahead video / audio after video
+ *
Each stage: +5ms and -15ms. audio ahead video / audio after video
+ *
+ *
+ *
+ * Recommendation of av pts time lead/lag at presentation:
+ *
+ *
TV: +15ms and -45ms. audio ahead video / audio after video.
+ *
Film: +22ms and -22ms. audio ahead video / audio after video.
+ *
+ *
+ *
+ * Maybe implemented as follows:
+ *
+ * d_av = vpts - apts;
+ * d_av < -22: audio after video == video ahead audio -> drop
+ * d_av > 22: audio ahead video == video after audio -> sleep(d_av - 10)
+ *
+ *
+ *
+ * Returns true if audio is ahead of video, otherwise false (video is ahead of audio).
+ * In case of the latter (false), the video frame shall be dropped!
+ *
+ * @param frame
+ * @return true if audio is ahead of video, otherwise false (video is ahead of audio)
+ */
+ protected boolean syncAV(int d_vpts) {
+ if( d_vpts > 22 ) {
+ if( DEBUG ) {
+ System.err.println("V (sleep): "+(d_vpts - 22 / 2)+" ms");
}
+ try {
+ Thread.sleep( d_vpts - 22 / 2 );
+ } catch (InterruptedException e) { }
}
- return lastFrame;
+ return true;
}
- protected abstract boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking);
- protected abstract void syncFrame2Audio(TextureFrame frame);
private final void newFrameAvailable(TextureFrame frame) {
- frameNumber++;
+ decodedFrameCount++;
synchronized(eventListenersLock) {
for(Iterator i = eventListeners.iterator(); i.hasNext(); ) {
i.next().newFrameAvailable(this, frame, System.currentTimeMillis());
@@ -500,6 +750,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
class FramePusher extends Thread {
private volatile boolean isRunning = false;
private volatile boolean isActive = false;
+ private volatile boolean isBlocked = false;
private volatile boolean shallPause = true;
private volatile boolean shallStop = false;
@@ -560,6 +811,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
public synchronized void doPause() {
if( isActive ) {
shallPause = true;
+ if( isBlocked && isActive ) {
+ this.interrupt();
+ }
while( isActive ) {
try {
this.wait(); // wait until paused
@@ -595,6 +849,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
public synchronized void doStop() {
if( isRunning ) {
shallStop = true;
+ if( isBlocked && isRunning ) {
+ this.interrupt();
+ }
while( isRunning ) {
this.notify(); // wake-up pause-block (opt)
try {
@@ -629,7 +886,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
try {
this.wait(); // wait until resumed
} catch (InterruptedException e) {
- e.printStackTrace();
+ if( !shallPause ) {
+ e.printStackTrace();
+ }
}
}
isActive = true;
@@ -639,23 +898,30 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
if( !shallStop ) {
TextureFrame nextFrame = null;
- boolean ok = false;
try {
- nextFrame = videoFramesFree.getBlocking(true /* clearRef */ );
+ isBlocked = true;
+ nextFrame = videoFramesFree.getBlocking(false /* clearRef */ );
+ isBlocked = false;
+ nextFrame.setPTS( TextureFrame.INVALID_PTS ); // mark invalid until processed!
if( getNextTextureImpl(gl, nextFrame, true) ) {
- gl.glFinish();
- videoFramesDecoded.putBlocking(nextFrame);
- newFrameAvailable(nextFrame);
- ok = true;
+ // gl.glFinish();
+ gl.glFlush(); // even better: sync object!
+ if( !videoFramesDecoded.put(nextFrame) ) {
+ throw new InternalError("XXX: "+GLMediaPlayerImpl.this);
+ }
+ final TextureFrame _nextFrame = nextFrame;
+ nextFrame = null;
+ newFrameAvailable(_nextFrame);
}
} catch (InterruptedException e) {
+ isBlocked = false;
if( !shallStop && !shallPause ) {
e.printStackTrace(); // oops
shallPause = false;
shallStop = true;
}
} finally {
- if( !ok && null != nextFrame ) { // put back
+ if( null != nextFrame ) { // put back
videoFramesFree.put(nextFrame);
}
}
@@ -689,10 +955,18 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
}
- protected final void updateAttributes(int width, int height, int bps_stream, int bps_video, int bps_audio,
- float fps, int totalFrames, int duration,
- String vcodec, String acodec) {
+ protected final void updateAttributes(int vid, int aid, int width, int height, int bps_stream,
+ int bps_video, int bps_audio, float fps,
+ int videoFrames, int audioFrames, int duration, String vcodec, String acodec) {
int event_mask = 0;
+ if( this.vid != vid ) {
+ event_mask |= GLMediaEventListener.EVENT_CHANGE_VID;
+ this.vid = vid;
+ }
+ if( this.aid != aid ) {
+ event_mask |= GLMediaEventListener.EVENT_CHANGE_AID;
+ this.aid = aid;
+ }
if( this.width != width || this.height != height ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_SIZE;
this.width = width;
@@ -701,6 +975,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
if( this.fps != fps ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_FPS;
this.fps = fps;
+ this.frame_period = (int) ( 1000f / fps + 0.5f );
}
if( this.bps_stream != bps_stream || this.bps_video != bps_video || this.bps_audio != bps_audio ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_BPS;
@@ -708,9 +983,10 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
this.bps_video = bps_video;
this.bps_audio = bps_audio;
}
- if( this.totalFrames != totalFrames || this.duration != duration ) {
+ if( this.videoFrames != videoFrames || this.audioFrames != audioFrames || this.duration != duration ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_LENGTH;
- this.totalFrames = totalFrames;
+ this.videoFrames = videoFrames;
+ this.audioFrames = audioFrames;
this.duration = duration;
}
if( (null!=acodec && acodec.length()>0 && !this.acodec.equals(acodec)) ) {
@@ -736,78 +1012,120 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
@Override
- public final synchronized State destroy(GL gl) {
- destroyFramePusher();
- destroyImpl(gl);
- removeAllTextureFrames(gl);
- state = State.Uninitialized;
- return state;
+ public final State destroy(GL gl) {
+ synchronized( stateLock ) {
+ destroyFramePusher();
+ destroyImpl(gl);
+ removeAllTextureFrames(gl);
+ state = State.Uninitialized;
+ return state;
+ }
}
protected abstract void destroyImpl(GL gl);
@Override
- public final synchronized URLConnection getURLConnection() {
+ public final URLConnection getURLConnection() {
return urlConn;
}
@Override
- public final synchronized String getVideoCodec() {
+ public final int getVID() { return vid; }
+
+ @Override
+ public final int getAID() { return aid; }
+
+ @Override
+ public final String getVideoCodec() {
return vcodec;
}
@Override
- public final synchronized String getAudioCodec() {
+ public final String getAudioCodec() {
return acodec;
}
@Override
- public final synchronized long getTotalFrames() {
- return totalFrames;
+ public final int getVideoFrames() {
+ return videoFrames;
+ }
+
+ public final int getAudioFrames() {
+ return audioFrames;
}
@Override
- public final synchronized int getDuration() {
+ public final int getDuration() {
return duration;
}
@Override
- public final synchronized long getStreamBitrate() {
+ public final long getStreamBitrate() {
return bps_stream;
}
@Override
- public final synchronized int getVideoBitrate() {
+ public final int getVideoBitrate() {
return bps_video;
}
@Override
- public final synchronized int getAudioBitrate() {
+ public final int getAudioBitrate() {
return bps_audio;
}
@Override
- public final synchronized float getFramerate() {
+ public final float getFramerate() {
return fps;
}
@Override
- public final synchronized int getWidth() {
+ public final int getWidth() {
return width;
}
@Override
- public final synchronized int getHeight() {
+ public final int getHeight() {
return height;
}
@Override
- public final synchronized String toString() {
- final float ct = getCurrentPosition() / 1000.0f, tt = getDuration() / 1000.0f;
+ public final String toString() {
+ final float tt = getDuration() / 1000.0f;
final String loc = ( null != urlConn ) ? urlConn.getURL().toExternalForm() : "" ;
- return "GLMediaPlayer["+state+", "+frameNumber+"/"+totalFrames+" frames, "+ct+"/"+tt+"s, speed "+playSpeed+", "+bps_stream+" bps, "+
- "Texture[count "+textureCount+", target "+toHexString(textureTarget)+", format "+toHexString(textureFormat)+", type "+toHexString(textureType)+"], "+
- "Stream[Video[<"+vcodec+">, "+width+"x"+height+", "+fps+" fps, "+bps_video+" bsp], "+
- "Audio[<"+acodec+">, "+bps_audio+" bsp]], "+loc+"]";
+ final int freeVideoFrames = null != videoFramesFree ? videoFramesFree.size() : 0;
+ final int decVideoFrames = null != videoFramesDecoded ? videoFramesDecoded.size() : 0;
+ return "GLMediaPlayer["+state+", frames[p "+presentedFrameCount+", d "+decodedFrameCount+", t "+videoFrames+" ("+tt+" s)], "+
+ "speed "+playSpeed+", "+bps_stream+" bps, "+
+ "Texture[count "+textureCount+", free "+freeVideoFrames+", dec "+decVideoFrames+", target "+toHexString(textureTarget)+", format "+toHexString(textureFormat)+", type "+toHexString(textureType)+"], "+
+ "Video[id "+vid+", <"+vcodec+">, "+width+"x"+height+", "+fps+" fps, "+bps_video+" bps], "+
+ "Audio[id "+aid+", <"+acodec+">, "+bps_audio+" bps, "+audioFrames+" frames], uri "+loc+"]";
+ }
+
+ @Override
+ public final String getPerfString() {
+ final int scr_pts = video_scr_pts +
+ (int) ( ( System.currentTimeMillis() - video_scr_t0 ) * playSpeed );
+ final int d_vpts = video_pts_last - scr_pts;
+ return getPerfStringImpl( scr_pts, video_pts_last, d_vpts, getVideoDPTSAvg() );
+ }
+ private final String getPerfStringImpl(final int scr_pts, final int video_pts, final int d_vpts, final int video_dpts_avg_diff) {
+ final float tt = getDuration() / 1000.0f;
+ final int audio_scr = (int) ( ( System.currentTimeMillis() - audio_scr_t0 ) * playSpeed );
+ final int audio_pts = getAudioPTSImpl();
+ final int d_apts = audio_pts - audio_scr;
+ final String audioSinkInfo;
+ final AudioSink audioSink = getAudioSink();
+ if( null != audioSink ) {
+ audioSinkInfo = "AudioSink[frames [d "+audioSink.getEnqueuedFrameCount()+", q "+audioSink.getQueuedFrameCount()+", f "+audioSink.getFreeFrameCount()+"], time "+audioSink.getQueuedTime()+", bytes "+audioSink.getQueuedByteCount()+"]";
+ } else {
+ audioSinkInfo = "";
+ }
+ final int freeVideoFrames = null != videoFramesFree ? videoFramesFree.size() : 0;
+ final int decVideoFrames = null != videoFramesDecoded ? videoFramesDecoded.size() : 0;
+ return state+", frames[p "+presentedFrameCount+", d "+decodedFrameCount+", t "+videoFrames+" ("+tt+" s)], "+
+ "speed " + playSpeed+", vSCR "+scr_pts+", vpts "+video_pts+", dSCR["+d_vpts+", avrg "+video_dpts_avg_diff+"], "+
+ "aSCR "+audio_scr+", apts "+audio_pts+" ( "+d_apts+" ), "+audioSinkInfo+
+ ", Texture[count "+textureCount+", free "+freeVideoFrames+", dec "+decVideoFrames+"]";
}
@Override
@@ -831,7 +1149,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
@Override
- public final synchronized GLMediaEventListener[] getEventListeners() {
+ public final GLMediaEventListener[] getEventListeners() {
synchronized(eventListenersLock) {
return eventListeners.toArray(new GLMediaEventListener[eventListeners.size()]);
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
index f1ce42257..5d70ca33d 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
@@ -38,6 +38,7 @@ import jogamp.opengl.util.av.GLMediaPlayerImpl;
import com.jogamp.common.nio.Buffers;
import com.jogamp.common.util.IOUtil;
+import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureData;
import com.jogamp.opengl.util.texture.TextureIO;
@@ -62,7 +63,7 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
}
@Override
- protected final boolean startImpl() {
+ protected final boolean playImpl() {
pos_start = (int)System.currentTimeMillis();
return true;
}
@@ -72,11 +73,6 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
return true;
}
- @Override
- protected final boolean stopImpl() {
- return true;
- }
-
@Override
protected final int seekImpl(int msec) {
pos_ms = msec;
@@ -86,20 +82,16 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
@Override
protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ nextFrame.setPTS( getAudioPTSImpl() );
return true;
}
- @Override
- protected final void syncFrame2Audio(TextureFrame frame) { }
@Override
- protected final int getCurrentPositionImpl() {
+ protected final int getAudioPTSImpl() {
pos_ms = (int)System.currentTimeMillis() - pos_start;
validatePos();
return pos_ms;
}
- @Override
- protected final int getAudioPTSImpl() { return getCurrentPositionImpl(); }
-
@Override
protected final void destroyImpl(GL gl) {
@@ -110,7 +102,7 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
}
@Override
- protected final void initGLStreamImpl(GL gl) throws IOException {
+ protected final void initGLStreamImpl(GL gl, int vid, int aid) throws IOException {
try {
URLConnection urlConn = IOUtil.getResource("jogl/util/data/av/test-ntsc01-160x90.png", this.getClass().getClassLoader());
if(null != urlConn) {
@@ -136,13 +128,14 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
GL.GL_RGBA, GL.GL_UNSIGNED_BYTE, false,
false, false, buffer, null);
}
+ final int r_aid = GLMediaPlayer.STREAM_ID_NONE == aid ? GLMediaPlayer.STREAM_ID_NONE : GLMediaPlayer.STREAM_ID_AUTO;
final float _fps = 24f;
final int _duration = 10*60*1000; // msec
final int _totalFrames = (int) ( (_duration/1000)*_fps );
- updateAttributes(_w, _h,
- 0, 0, 0,
- _fps, _totalFrames, _duration,
- "png-static", null);
+ updateAttributes(GLMediaPlayer.STREAM_ID_AUTO, r_aid,
+ _w, _h, 0,
+ 0, 0, _fps,
+ _totalFrames, 0, _duration, "png-static", null);
}
@Override
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
index 83a5960f1..dc7ceae39 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
@@ -42,9 +42,11 @@ import com.jogamp.common.util.VersionNumber;
import com.jogamp.gluegen.runtime.ProcAddressTable;
import com.jogamp.opengl.util.GLPixelStorageModes;
import com.jogamp.opengl.util.av.AudioSink;
+import com.jogamp.opengl.util.av.AudioSink.AudioDataFormat;
+import com.jogamp.opengl.util.av.AudioSink.AudioDataType;
import com.jogamp.opengl.util.av.AudioSinkFactory;
+import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
-import com.jogamp.opengl.util.texture.TextureSequence.TextureFrame;
import jogamp.opengl.GLContextImpl;
import jogamp.opengl.util.av.GLMediaPlayerImpl;
@@ -136,13 +138,17 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
( vers >> 8 ) & 0xFF,
( vers >> 0 ) & 0xFF );
}
+
+ //
+ // General
+ //
+
+ protected long moviePtr = 0;
//
// Video
//
- protected long moviePtr = 0;
- protected GLPixelStorageModes psm;
protected PixelFormat vPixelFmt = null;
protected int vPlanes = 0;
protected int vBitsPerPixel = 0;
@@ -152,15 +158,17 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
protected int texWidth, texHeight; // overall (stuffing planes in one texture)
protected ByteBuffer texCopy;
protected String singleTexComp = "r";
+ protected GLPixelStorageModes psm;
//
// Audio
//
- protected final int AudioFrameCount = 8;
- protected final AudioSink audioSink;
- protected final int maxAvailableAudio;
- protected AudioSink.AudioDataFormat chosenAudioFormat;
+ protected static final int AFRAMES_PER_VFRAME = 8;
+ protected int aFrameCount = 0;
+ protected SampleFormat aSampleFmt = null;
+ protected AudioSink.AudioDataFormat avChosenAudioFormat;
+ protected AudioSink.AudioDataFormat sinkChosenAudioFormat;
public FFMPEGMediaPlayer() {
if(!available) {
@@ -171,12 +179,11 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
throw new GLException("Couldn't create FFMPEGInstance");
}
psm = new GLPixelStorageModes();
- audioSink = AudioSinkFactory.createDefault();
- maxAvailableAudio = audioSink.getQueuedByteCount();
+ audioSink = null;
}
@Override
protected final int validateTextureCount(int desiredTextureCount) {
- return desiredTextureCount>1 ? desiredTextureCount : 2;
+ return desiredTextureCount>2 ? Math.max(4, desiredTextureCount) : 2;
}
@Override
protected final boolean requiresOffthreadGLCtx() { return true; }
@@ -187,10 +194,18 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
destroyInstance0(moviePtr);
moviePtr = 0;
}
+ destroyAudioSink();
+ }
+ private final void destroyAudioSink() {
+ final AudioSink _audioSink = audioSink;
+ if( null != _audioSink ) {
+ audioSink = null;
+ _audioSink.destroy();
+ }
}
@Override
- protected final void initGLStreamImpl(GL gl) throws IOException {
+ protected final void initGLStreamImpl(GL gl, int vid, int aid) throws IOException {
if(0==moviePtr) {
throw new GLException("FFMPEG native instance null");
}
@@ -209,11 +224,32 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
}
final String urlS=urlConn.getURL().toExternalForm();
+
+ aFrameCount = AFRAMES_PER_VFRAME * textureCount + AFRAMES_PER_VFRAME/2;
- chosenAudioFormat = audioSink.initSink(audioSink.getPreferredFormat(), AudioFrameCount);
System.err.println("setURL: p1 "+this);
- setStream0(moviePtr, urlS, -1, -1, AudioFrameCount); // issues updateAttributes*(..)
- System.err.println("setURL: p2 "+this);
+ destroyAudioSink();
+ AudioSink _audioSink;
+ if( GLMediaPlayer.STREAM_ID_NONE == aid ) {
+ _audioSink = AudioSinkFactory.createNull();
+ } else {
+ _audioSink = AudioSinkFactory.createDefault();
+ }
+ final AudioDataFormat preferredAudioFormat = _audioSink.getPreferredFormat();
+ // setStream(..) issues updateAttributes*(..), and defines avChosenAudioFormat, vid, aid, .. etc
+ setStream0(moviePtr, urlS, vid, aid, aFrameCount, preferredAudioFormat.channelCount, preferredAudioFormat.sampleRate);
+ // final int audioBytesPerFrame = bps_audio/8000 * frame_period * textureCount;
+
+ System.err.println("setURL: p2 preferred "+preferredAudioFormat+", avChosen "+avChosenAudioFormat+", "+this);
+ sinkChosenAudioFormat = _audioSink.initSink(avChosenAudioFormat, aFrameCount);
+ System.err.println("setURL: p3 avChosen "+avChosenAudioFormat+", chosen "+sinkChosenAudioFormat);
+ if( null == sinkChosenAudioFormat ) {
+ System.err.println("AudioSink "+_audioSink.getClass().getName()+" does not support "+avChosenAudioFormat+", using Null");
+ _audioSink.destroy();
+ _audioSink = AudioSinkFactory.createNull();
+ sinkChosenAudioFormat = _audioSink.initSink(avChosenAudioFormat, aFrameCount);
+ }
+ audioSink = _audioSink;
int tf, tif=GL.GL_RGBA; // texture format and internal format
switch(vBytesPerPixelPerPlane) {
@@ -256,7 +292,8 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
private void updateAttributes2(int pixFmt, int planes, int bitsPerPixel, int bytesPerPixelPerPlane,
int lSz0, int lSz1, int lSz2,
- int tWd0, int tWd1, int tWd2) {
+ int tWd0, int tWd1, int tWd2,
+ int sampleFmt, int sampleRate, int channels) {
vPixelFmt = PixelFormat.valueOf(pixFmt);
vPlanes = planes;
vBitsPerPixel = bitsPerPixel;
@@ -286,12 +323,53 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
default: // FIXME: Add more planar formats !
throw new RuntimeException("Unsupported pixelformat: "+vPixelFmt);
}
+
+ aSampleFmt = SampleFormat.valueOf(sampleFmt);
+ final int sampleSize;
+ final boolean signed, fixedP;
+ switch( aSampleFmt ) {
+ case S32:
+ case S32P:
+ sampleSize = 32;
+ signed = true;
+ fixedP = true;
+ break;
+ case S16:
+ case S16P:
+ sampleSize = 16;
+ signed = true;
+ fixedP = true;
+ break;
+ case U8:
+ case U8P:
+ sampleSize = 8;
+ signed = false;
+ fixedP = true;
+ break;
+ case DBL:
+ case DBLP:
+ sampleSize = 64;
+ signed = true;
+ fixedP = true;
+ break;
+ case FLT:
+ case FLTP:
+ sampleSize = 32;
+ signed = true;
+ fixedP = true;
+ break;
+ default: // FIXME: Add more planar formats !
+ throw new RuntimeException("Unsupported sampleformat: "+aSampleFmt);
+ }
+ avChosenAudioFormat = new AudioDataFormat(AudioDataType.PCM, sampleRate, sampleSize, channels, signed, fixedP, true /* littleEndian */);
+
if(DEBUG) {
- System.err.println("XXX0: fmt "+vPixelFmt+", planes "+vPlanes+", bpp "+vBitsPerPixel+"/"+vBytesPerPixelPerPlane);
+ System.err.println("audio: fmt "+aSampleFmt+", "+avChosenAudioFormat);
+ System.err.println("video: fmt "+vPixelFmt+", planes "+vPlanes+", bpp "+vBitsPerPixel+"/"+vBytesPerPixelPerPlane);
for(int i=0; i<3; i++) {
- System.err.println("XXX0 "+i+": "+vTexWidth[i]+"/"+vLinesize[i]);
+ System.err.println("video: "+i+": "+vTexWidth[i]+"/"+vLinesize[i]);
}
- System.err.println("XXX0 total tex "+texWidth+"x"+texHeight);
+ System.err.println("video: total tex "+texWidth+"x"+texHeight);
}
}
@@ -355,54 +433,27 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
}
@Override
- protected final synchronized int getCurrentPositionImpl() {
- return 0!=moviePtr ? getVideoPTS0(moviePtr) : 0;
- }
-
- @Override
- public final int getAudioPTSImpl() { return 0; }
-
- @Override
- protected final synchronized boolean setPlaySpeedImpl(float rate) {
- return true;
- }
-
- @Override
- public final synchronized boolean startImpl() {
+ public final boolean playImpl() {
if(0==moviePtr) {
return false;
}
return true;
}
- /** @return time position after issuing the command */
@Override
- public final synchronized boolean pauseImpl() {
+ public final boolean pauseImpl() {
if(0==moviePtr) {
return false;
}
return true;
}
- /** @return time position after issuing the command */
- @Override
- public final synchronized boolean stopImpl() {
- if(0==moviePtr) {
- return false;
- }
- return true;
- }
-
- /** @return time position after issuing the command */
@Override
protected final synchronized int seekImpl(int msec) {
if(0==moviePtr) {
throw new GLException("FFMPEG native instance null");
}
- int pts0 = getVideoPTS0(moviePtr);
- int pts1 = seek0(moviePtr, msec);
- System.err.println("Seek: "+pts0+" -> "+msec+" : "+pts1);
- return pts1;
+ return seek0(moviePtr, msec);
}
@Override
@@ -427,7 +478,6 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
psm.restore(gl);
}
if( 0 < avPTS ) {
- vSTS = avPTS;
nextFrame.setPTS(avPTS);
return true;
} else {
@@ -436,50 +486,15 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
}
private final void pushSound(ByteBuffer sampleData, int data_size, int audio_pts) {
- aSTS = audio_pts;
- final AudioSink.AudioFrame frame = new AudioSink.AudioFrame(sampleData, data_size, audio_pts);
- if( audioSink.isDataAvailable(frame.dataSize) ) {
- audioSink.writeData(frame);
+ setFirstAudioPTS2SCR( audio_pts );
+ if( 1.0f == playSpeed || audioSinkPlaySpeedSet ) {
+ audioSink.enqueueData( new AudioSink.AudioFrame(sampleData, data_size, audio_pts ) );
}
}
-
- /** last audio streaming TS */
- private int aSTS = 0;
- /** last video streaming TS */
- private int vSTS = 0;
-
- private long lastAudioTime = 0;
- private static final int audio_dt_d = 400;
- private long lastVideoTime = 0;
- private static final int video_dt_d = 9;
-
+
@Override
- protected final void syncFrame2Audio(TextureFrame frame) {
- /**
- // poor mans video sync .. TODO: off thread 'readNextPackage0(..)' on shared GLContext and multi textures/unit!
- final long now = System.currentTimeMillis();
- // Try sync video to audio
- final long now_d = now - lastAudioTime;
- final long pts_d = vSTS - aSTS - 444; // hack 444 == play video 444ms ahead of audio
- final long dt = Math.min(47, (long) ( (float) ( pts_d - now_d ) / getPlaySpeed() ) ) ;
- //final long dt = (long) ( (float) ( pts_d - now_d ) / getPlaySpeed() ) ;
- final boolean sleep = dt>video_dt_d && dt<1000 && audioSink.getQueuedByteCount()
+ * Always uses {@link AudioSink.AudioDataFormat}:
+ *
* Audio maybe supported and played back internally or via an {@link AudioSink} implementation,
* if an audio stream is selected in {@link #initGLStream(GL, int, URLConnection, int, int)}.
- *
@@ -91,14 +92,47 @@ import com.jogamp.opengl.util.texture.TextureSequence;
* Milliseconds granularity is also more than enough to deal with A-V synchronization,
* where the threshold usually lies within 100ms.
*
+ *
+ *
+ * The class follows a passive A/V synchronization pattern.
+ * Audio is being untouched, while {@link #getNextTexture(GL, boolean)} delivers a new video frame
+ * only, if its timestamp is less than 22ms ahead of time.
+ * Otherwise the early frame is cached for later retrieval and the previous frame is returned.
+ * FIXME: Refine!
+ *
+ * Recommendation of audio/video pts time lead/lag at production:
+ *
+ *
Overall: +40ms and -60ms audio ahead video / audio after video
+ *
Each stage: +5ms and -15ms. audio ahead video / audio after video
+ *
+ *
+ *
+ * Recommendation of av pts time lead/lag at presentation:
+ *
+ *
TV: +15ms and -45ms. audio ahead video / audio after video.
+ *
Film: +22ms and -22ms. audio ahead video / audio after video.
+ *
+ *
*/
public interface GLMediaPlayer extends TextureSequence {
public static final boolean DEBUG = Debug.debug("GLMediaPlayer");
+ public static final boolean DEBUG_NATIVE = Debug.debug("GLMediaPlayer.Native");
/** Constant {@value} for mute or not available. See Audio and video Stream IDs. */
public static final int STREAM_ID_NONE = -2;
/** Constant {@value} for auto or unspecified. See Audio and video Stream IDs. */
public static final int STREAM_ID_AUTO = -1;
+
+ /** Maximum video frame async .. */
+ public static final int MAXIMUM_VIDEO_ASYNC = 22;
public interface GLMediaEventListener extends TexSeqEventListener {
@@ -228,13 +262,14 @@ public interface GLMediaPlayer extends TextureSequence {
public int getAID();
/**
- * @return the current decoded frame count since {@link #initGLStream(GL, int, URLConnection, int, int)}.
+ * @return the current decoded frame count since {@link #play()} and {@link #seek(int)}
+ * as increased by {@link #getNextTexture(GL, boolean)} or the decoding thread.
*/
public int getDecodedFrameCount();
/**
- * @return the current presented frame count since {@link #initGLStream(GL, int, URLConnection, int, int)},
- * increased by {@link #getNextTexture(GL, boolean)}.
+ * @return the current presented frame count since {@link #play()} and {@link #seek(int)}
+ * as increased by {@link #getNextTexture(GL, boolean)} for new frames.
*/
public int getPresentedFrameCount();
@@ -250,6 +285,9 @@ public interface GLMediaPlayer extends TextureSequence {
/**
* {@inheritDoc}
+ *
*
* @see #addEventListener(GLMediaEventListener)
* @see GLMediaEventListener#newFrameAvailable(GLMediaPlayer, TextureFrame, long)
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 50801e791..05fda99ae 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -110,23 +110,31 @@ public interface TextureSequence {
* to associated related data.
*/
public static class TextureFrame {
- /** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE {@value}. */
- public static final int INVALID_PTS = Integer.MIN_VALUE;
+ /** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE 0x80000000 {@value}. */
+ public static final int INVALID_PTS = 0x80000000 ; // == -2147483648 == Integer.MIN_VALUE;
public TextureFrame(Texture t) {
texture = t;
pts = INVALID_PTS;
+ duration = 0;
}
public final Texture getTexture() { return texture; }
+ /** Get this frame's presentation timestamp (PTS) in milliseconds. */
public final int getPTS() { return pts; }
+ /** Set this frame's presentation timestamp (PTS) in milliseconds. */
public final void setPTS(int pts) { this.pts = pts; }
+ /** Get this frame's duration in milliseconds. */
+ public final int getDuration() { return duration; }
+ /** Set this frame's duration in milliseconds. */
+ public final void setDuration(int duration) { this.duration = duration; }
public String toString() {
- return "TextureFrame[" + pts + "ms: " + texture + "]";
+ return "TextureFrame[pts " + pts + " ms, l " + duration + " ms, "+ texture + "]";
}
protected final Texture texture;
protected int pts;
+ protected int duration;
}
public interface TexSeqEventListener {
diff --git a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
index e14642c34..86e6bc121 100644
--- a/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
+++ b/src/jogl/classes/jogamp/opengl/android/av/AndroidGLMediaPlayerAPI14.java
@@ -222,7 +222,7 @@ public class AndroidGLMediaPlayerAPI14 extends GLMediaPlayerImpl {
}
@Override
- protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking, boolean issuePreAndPost) {
if(null != stex && null != mp) {
final SurfaceTextureFrame nextSFrame = (SurfaceTextureFrame) nextFrame;
final Surface nextSurface = nextSFrame.getSurface();
diff --git a/src/jogl/classes/jogamp/opengl/openal/av/ALAudioSink.java b/src/jogl/classes/jogamp/opengl/openal/av/ALAudioSink.java
index 5783c32f1..217ab2954 100644
--- a/src/jogl/classes/jogamp/opengl/openal/av/ALAudioSink.java
+++ b/src/jogl/classes/jogamp/opengl/openal/av/ALAudioSink.java
@@ -80,7 +80,7 @@ public class ALAudioSink implements AudioSink {
private SyncedRingbuffer alBufferAvail = null;
private SyncedRingbuffer alBufferPlaying = null;
private volatile int alBufferBytesQueued = 0;
- private volatile int ptsPlaying = 0;
+ private volatile int playingPTS = AudioFrame.INVALID_PTS;
private volatile int enqueuedFrameCount;
private int[] alSource = null;
@@ -207,11 +207,11 @@ public class ALAudioSink implements AudioSink {
return "ALAudioSink[init "+initialized+", playRequested "+playRequested+", device "+deviceSpecifier+", ctx "+toHexString(ctxHash)+", alSource "+alSrcName+
", chosen "+chosenFormat+", alFormat "+toHexString(alFormat)+
", playSpeed "+playSpeed+", buffers[total "+alBuffersLen+", avail "+alBufferAvail.size()+", "+
- "queued["+alBufferPlaying.size()+", apts "+ptsPlaying+", "+getQueuedTime() + " ms, " + alBufferBytesQueued+" bytes]";
+ "queued["+alBufferPlaying.size()+", apts "+getPTS()+", "+getQueuedTime() + " ms, " + alBufferBytesQueued+" bytes]";
}
public final String getPerfString() {
final int alBuffersLen = null != alBuffers ? alBuffers.length : 0;
- return "Play [buffer "+alBufferPlaying.size()+"/"+alBuffersLen+", apts "+ptsPlaying+", "+getQueuedTime() + " ms, " + alBufferBytesQueued+" bytes]";
+ return "Play [buffer "+alBufferPlaying.size()+"/"+alBuffersLen+", apts "+getPTS()+", "+getQueuedTime() + " ms, " + alBufferBytesQueued+" bytes]";
}
@Override
@@ -289,9 +289,9 @@ public class ALAudioSink implements AudioSink {
t.printStackTrace();
}
}
- alBufferAvail.clear(true);
+ alBufferAvail.clear();
alBufferAvail = null;
- alBufferPlaying.clear(true);
+ alBufferPlaying.clear();
alBufferPlaying = null;
alBufferBytesQueued = 0;
alBuffers = null;
@@ -434,7 +434,11 @@ public class ALAudioSink implements AudioSink {
}
final int dequeuedBufferCount = dequeueBuffer( false /* all */, wait );
final ActiveBuffer currentBuffer = alBufferPlaying.peek();
- ptsPlaying = null != currentBuffer ? currentBuffer.pts : audioFrame.pts;
+ if( null != currentBuffer ) {
+ playingPTS = currentBuffer.pts;
+ } else {
+ playingPTS = audioFrame.pts;
+ }
if( DEBUG ) {
System.err.println(getThreadName()+": ALAudioSink: Write "+audioFrame.pts+", "+getQueuedTimeImpl(audioFrame.dataSize)+" ms, dequeued "+dequeuedBufferCount+", wait "+wait+", "+getPerfString());
}
@@ -652,5 +656,5 @@ public class ALAudioSink implements AudioSink {
}
@Override
- public final int getPTS() { return ptsPlaying; }
+ public final int getPTS() { return playingPTS; }
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
index c1cfc0d95..85b599c0e 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java
@@ -42,7 +42,9 @@ import javax.media.opengl.GLES2;
import javax.media.opengl.GLException;
import javax.media.opengl.GLProfile;
+import com.jogamp.common.os.Platform;
import com.jogamp.opengl.util.av.AudioSink;
+import com.jogamp.opengl.util.av.AudioSink.AudioFrame;
import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureSequence;
@@ -92,9 +94,10 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected int width = 0;
/** Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int height = 0;
- /** Video fps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ /** Video avg. fps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected float fps = 0;
- protected int frame_period = 0;
+ /** Video avg. frame duration in ms. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
+ protected float frame_duration = 0f;
/** Stream bps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
protected int bps_stream = 0;
/** Video bps. Shall be set by the {@link #initGLStreamImpl(GL, int, int)} method implementation. */
@@ -114,6 +117,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected volatile int decodedFrameCount = 0;
protected int presentedFrameCount = 0;
+ protected int displayedFrameCount = 0;
protected volatile int video_pts_last = 0;
/** See {@link #getAudioSink()}. Set by implementation if used from within {@link #initGLStreamImpl(GL, int, int)}! */
@@ -139,17 +143,8 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
/** Maximum valid video pts diff. */
private static final int VIDEO_DPTS_MAX = 5000; // 5s max diff
/** Trigger video PTS reset with given cause as bitfield. */
- private volatile int videoSCR_reset = 0;
+ private boolean videoSCR_reset = false;
- private final boolean isSCRCause(int bit) { return 0 != ( bit & videoSCR_reset); }
- /** SCR reset due to: Start, Resume, Seek, .. */
- private static final int SCR_RESET_FORCE = 1 << 0;
- /** SCR reset due to: PlaySpeed */
- private static final int SCR_RESET_SPEED = 1 << 1;
-
- /** Latched video PTS reset, to wait until valid pts after invalidation of cached ones. Currently [1..{@link #VIDEO_DPTS_NUM}] frames. */
- private int videoSCR_reset_latch = 0;
-
protected SyncedRingbuffer videoFramesFree = null;
protected SyncedRingbuffer videoFramesDecoded = null;
protected volatile TextureFrame lastFrame = null;
@@ -279,16 +274,17 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
switch( state ) {
case Paused:
if( playImpl() ) {
- resetAudioVideoSCR(SCR_RESET_FORCE);
- resumeFramePusher();
+ // FIXME
+ resetAudioVideoPTS();
if( null != audioSink ) {
- audioSink.play();
- }
+ audioSink.play(); // cont. w/ new data
+ }
+ resumeFramePusher();
state = State.Playing;
}
default:
}
- if(DEBUG) { System.err.println("Start: "+toString()); }
+ if(DEBUG) { System.err.println("Play: "+toString()); }
return state;
}
}
@@ -297,16 +293,15 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
public final State pause() {
synchronized( stateLock ) {
if( State.Playing == state ) {
- State _state = state;
state = State.Paused;
- if( pauseImpl() ) {
- _state = State.Paused;
- pauseFramePusher();
- if( null != audioSink ) {
- audioSink.pause();
- }
+ // FIXME
+ pauseFramePusher();
+ if( null != audioSink ) {
+ audioSink.pause();
+ }
+ if( !pauseImpl() ) {
+ play();
}
- state = _state;
}
if(DEBUG) { System.err.println("Pause: "+toString()); }
return state;
@@ -322,14 +317,12 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
case Paused:
final State _state = state;
state = State.Paused;
+ // FIXME
pauseFramePusher();
- resetAudioVideoSCR(SCR_RESET_FORCE);
pts1 = seekImpl(msec);
- if( null != audioSink ) {
- audioSink.flush();
- if( State.Playing == _state ) {
- audioSink.play(); // cont. w/ new data
- }
+ resetAllAudioVideoSync();
+ if( null != audioSink && State.Playing == _state ) {
+ audioSink.play(); // cont. w/ new data
}
resumeFramePusher();
state = _state;
@@ -358,7 +351,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
rate = 1.0f;
}
if( setPlaySpeedImpl(rate) ) {
- resetAudioVideoSCR(SCR_RESET_SPEED);
+ resetAudioVideoPTS();
playSpeed = rate;
if(DEBUG) { System.err.println("SetPlaySpeed: "+toString()); }
res = true;
@@ -392,6 +385,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
decodedFrameCount = 0;
presentedFrameCount = 0;
+ displayedFrameCount = 0;
this.urlConn = urlConn;
if (this.urlConn != null) {
try {
@@ -405,7 +399,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
videoFramesFree = new SyncedRingbuffer(createTexFrames(gl, textureCount), true /* full */);
if( TEXTURE_COUNT_DEFAULT < textureCount ) {
videoFramesDecoded = new SyncedRingbuffer(new TextureFrame[textureCount], false /* full */);
- framePusher = new FramePusher(gl, requiresOffthreadGLCtx());
+ framePusher = new FramePusher(gl);
framePusher.doStart();
} else {
videoFramesDecoded = null;
@@ -448,7 +442,6 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected int validateTextureCount(int desiredTextureCount) {
return TEXTURE_COUNT_DEFAULT;
}
- protected boolean requiresOffthreadGLCtx() { return false; }
private final TextureFrame[] createTexFrames(GL gl, final int count) {
final int[] texNames = new int[count];
@@ -543,6 +536,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
textureCount=0;
}
+ protected TextureFrame cachedFrame = null;
+ protected long lastTimeMillis = 0;
+
@Override
public final TextureFrame getNextTexture(GL gl, boolean blocking) throws IllegalStateException {
synchronized( stateLock ) {
@@ -554,68 +550,95 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
boolean ok = true;
boolean dropFrame = false;
try {
- do {
- if( TEXTURE_COUNT_DEFAULT < textureCount ) {
+ do {
+ final long currentTimeMillis;
+ final boolean playCached = null != cachedFrame;
+ if( dropFrame ) {
+ presentedFrameCount--;
+ dropFrame = false;
+ }
+ if( playCached ) {
+ nextFrame = cachedFrame;
+ cachedFrame = null;
+ presentedFrameCount--;
+ currentTimeMillis = Platform.currentTimeMillis();
+ } else if( TEXTURE_COUNT_DEFAULT < textureCount ) {
nextFrame = videoFramesDecoded.getBlocking(false /* clearRef */ );
+ currentTimeMillis = Platform.currentTimeMillis();
} else {
nextFrame = videoFramesFree.getBlocking(false /* clearRef */ );
- if( getNextTextureImpl(gl, nextFrame, blocking) ) {
- newFrameAvailable(nextFrame);
- } else {
- ok = false;
+ nextFrame.setPTS( TextureFrame.INVALID_PTS ); // mark invalid until processed!
+ ok = getNextTextureImpl(gl, nextFrame, blocking, true /* issuePreAndPost */);
+ currentTimeMillis = Platform.currentTimeMillis();
+ if( ok ) {
+ newFrameAvailable(nextFrame, currentTimeMillis);
}
}
+ if( DEBUG ) {
+ System.err.println("> "+currentTimeMillis+", d "+(currentTimeMillis-lastTimeMillis)+", playCached "+playCached);
+ }
if( ok ) {
presentedFrameCount++;
- final int video_pts;
- if( 0 != videoSCR_reset ) {
- if( isSCRCause(SCR_RESET_FORCE) ) {
- videoSCR_reset_latch = VIDEO_DPTS_NUM / 2;
- resetVideoDPTS();
- resetAllVideoPTS();
+ final int video_pts = nextFrame.getPTS();
+ if( video_pts != TextureFrame.INVALID_PTS ) {
+ lastTimeMillis = currentTimeMillis;
+
+ final int audio_pts = getAudioPTSImpl();
+ final int audio_scr = (int) ( ( currentTimeMillis - audio_scr_t0 ) * playSpeed );
+ final int d_apts;
+ if( audio_pts != AudioFrame.INVALID_PTS ) {
+ d_apts = audio_pts - audio_scr;
} else {
- // SCR_RESET_SPEED
- videoSCR_reset_latch = 1;
+ d_apts = 0;
}
- videoSCR_reset = 0;
- video_pts = TextureFrame.INVALID_PTS;
- } else {
- video_pts = nextFrame.getPTS();
- }
- if( video_pts != TextureFrame.INVALID_PTS ) {
+
final int frame_period_last = video_pts - video_pts_last; // rendering loop interrupted ?
- if( videoSCR_reset_latch > 0 || frame_period_last > frame_period*10 ) {
- if( videoSCR_reset_latch > 0 ) {
- videoSCR_reset_latch--;
- }
- setFirstVideoPTS2SCR( video_pts );
+ if( videoSCR_reset || frame_period_last > frame_duration*10 ) {
+ videoSCR_reset = false;
+ video_scr_t0 = currentTimeMillis;
+ video_scr_pts = video_pts;
}
- final int scr_pts = video_scr_pts +
- (int) ( ( System.currentTimeMillis() - video_scr_t0 ) * playSpeed );
- final int d_vpts = video_pts - scr_pts;
+ final int video_scr = video_scr_pts + (int) ( ( currentTimeMillis - video_scr_t0 ) * playSpeed );
+ final int d_vpts = video_pts - video_scr;
+ // final int d_avpts = d_vpts - d_apts;
if( -VIDEO_DPTS_MAX > d_vpts || d_vpts > VIDEO_DPTS_MAX ) {
+ // if( -VIDEO_DPTS_MAX > d_avpts || d_avpts > VIDEO_DPTS_MAX ) {
if( DEBUG ) {
- System.err.println( getPerfStringImpl( scr_pts, video_pts, d_vpts, 0 ) );
+ System.err.println( "AV*: "+getPerfStringImpl( video_scr, video_pts, d_vpts, audio_scr, audio_pts, d_apts, 0 ) + ", "+nextFrame+", playCached " + playCached+ ", dropFrame "+dropFrame);
}
} else {
+ final int dpy_den = displayedFrameCount > 0 ? displayedFrameCount : 1;
+ final int avg_dpy_duration = ( (int) ( currentTimeMillis - video_scr_t0 ) ) / dpy_den ; // ms/f
+ final int maxVideoDelay = Math.min(avg_dpy_duration, MAXIMUM_VIDEO_ASYNC);
video_dpts_count++;
+ // video_dpts_cum = d_avpts + VIDEO_DPTS_COEFF * video_dpts_cum;
video_dpts_cum = d_vpts + VIDEO_DPTS_COEFF * video_dpts_cum;
- final int video_dpts_avg_diff = getVideoDPTSAvg();
- if( DEBUG ) {
- System.err.println( getPerfStringImpl( scr_pts, video_pts, d_vpts, video_dpts_avg_diff ) );
- }
- if( blocking && syncAVRequired() ) {
- if( !syncAV( (int) ( video_dpts_avg_diff / playSpeed + 0.5f ) ) ) {
- resetVideoDPTS();
- dropFrame = true;
- }
+ final int video_dpts_avg_diff = video_dpts_count >= VIDEO_DPTS_NUM ? getVideoDPTSAvg() : 0;
+ final int dt = (int) ( video_dpts_avg_diff / playSpeed + 0.5f );
+ // final int dt = (int) ( d_vpts / playSpeed + 0.5f );
+ // final int dt = (int) ( d_avpts / playSpeed + 0.5f );
+ if( dt > maxVideoDelay ) {
+ cachedFrame = nextFrame;
+ nextFrame = null;
+ } else if ( dt < -maxVideoDelay ) {
+ dropFrame = true;
}
video_pts_last = video_pts;
+ if( DEBUG ) {
+ System.err.println( "AV_: "+getPerfStringImpl( video_scr, video_pts, d_vpts,
+ audio_scr, audio_pts, d_apts,
+ video_dpts_avg_diff ) +
+ ", avg dpy-fps "+avg_dpy_duration+" ms/f, maxD "+maxVideoDelay+" ms, "+nextFrame+", playCached " + playCached + ", dropFrame "+dropFrame);
+ }
}
+ } else if( DEBUG ) {
+ System.err.println("Invalid PTS: "+nextFrame);
+ }
+ if( null != nextFrame ) {
+ final TextureFrame _lastFrame = lastFrame;
+ lastFrame = nextFrame;
+ videoFramesFree.putBlocking(_lastFrame);
}
- final TextureFrame _lastFrame = lastFrame;
- lastFrame = nextFrame;
- videoFramesFree.putBlocking(_lastFrame);
}
} while( dropFrame );
} catch (InterruptedException e) {
@@ -623,14 +646,19 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
e.printStackTrace();
} finally {
if( !ok && null != nextFrame ) { // put back
- videoFramesFree.put(nextFrame);
+ if( !videoFramesFree.put(nextFrame) ) {
+ throw new InternalError("XXX: free "+videoFramesFree+", decoded "+videoFramesDecoded+", "+GLMediaPlayerImpl.this);
+ }
}
}
}
+ displayedFrameCount++;
return lastFrame;
}
}
- protected abstract boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking);
+ protected void preNextTextureImpl(GL gl) {}
+ protected void postNextTextureImpl(GL gl) {}
+ protected abstract boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking, boolean issuePreAndPost);
protected boolean syncAVRequired() { return false; }
/**
@@ -654,95 +682,49 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
*/
protected void setFirstAudioPTS2SCR(int pts) {
if( audioSCR_reset ) {
- audio_scr_t0 = System.currentTimeMillis() - pts;
+ audio_scr_t0 = Platform.currentTimeMillis() - pts;
audioSCR_reset = false;
}
}
- private void setFirstVideoPTS2SCR(int pts) {
- // video_scr_t0 = System.currentTimeMillis() - pts;
- video_scr_t0 = System.currentTimeMillis();
- video_scr_pts = pts;
- }
- private void resetAllVideoPTS() {
+ private void flushAllVideoFrames() {
if( null != videoFramesFree ) {
- final TextureFrame[] texFrames = videoFramesFree.getArray();
- for(int i=0; i
- * https://en.wikipedia.org/wiki/Audio_to_video_synchronization
- *
- * d_av = v_pts - a_pts;
- *
- *
- *
- * Recommendation of audio/video pts time lead/lag at production:
- *
- *
Overall: +40ms and -60ms audio ahead video / audio after video
- *
Each stage: +5ms and -15ms. audio ahead video / audio after video
- *
- *
- *
- * Recommendation of av pts time lead/lag at presentation:
- *
- *
TV: +15ms and -45ms. audio ahead video / audio after video.
- *
Film: +22ms and -22ms. audio ahead video / audio after video.
- *
- *
- *
- * Maybe implemented as follows:
- *
- * d_av = vpts - apts;
- * d_av < -22: audio after video == video ahead audio -> drop
- * d_av > 22: audio ahead video == video after audio -> sleep(d_av - 10)
- *
- *
- *
- * Returns true if audio is ahead of video, otherwise false (video is ahead of audio).
- * In case of the latter (false), the video frame shall be dropped!
- *
- * @param frame
- * @return true if audio is ahead of video, otherwise false (video is ahead of audio)
- */
- protected boolean syncAV(int d_vpts) {
- if( d_vpts > 22 ) {
- if( DEBUG ) {
- System.err.println("V (sleep): "+(d_vpts - 22 / 2)+" ms");
- }
- try {
- Thread.sleep( d_vpts - 22 / 2 );
- } catch (InterruptedException e) { }
- }
- return true;
+ private final int getVideoDPTSAvg() {
+ return (int) ( video_dpts_cum * (1.0f - VIDEO_DPTS_COEFF) + 0.5f );
}
- private final void newFrameAvailable(TextureFrame frame) {
- decodedFrameCount++;
+ private final void newFrameAvailable(TextureFrame frame, long currentTimeMillis) {
+ decodedFrameCount++;
+ if( 0 == frame.getDuration() ) { // patch frame duration if not set already
+ frame.setDuration( (int) frame_duration );
+ }
synchronized(eventListenersLock) {
for(Iterator i = eventListeners.iterator(); i.hasNext(); ) {
- i.next().newFrameAvailable(this, frame, System.currentTimeMillis());
+ i.next().newFrameAvailable(this, frame, currentTimeMillis);
}
}
}
@@ -759,12 +741,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
private GLDrawable dummyDrawable = null;
private GLContext sharedGLCtx = null;
- FramePusher(GL gl, boolean createSharedCtx) {
+ FramePusher(GL gl) {
setDaemon(true);
- this.gl = createSharedCtx ? createSharedGL(gl) : gl;
- }
-
- private GL createSharedGL(GL gl) {
+
final GLContext glCtx = gl.getContext();
final boolean glCtxCurrent = glCtx.isCurrent();
final GLProfile glp = gl.getGLProfile();
@@ -779,8 +758,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
} else {
sharedGLCtx.release();
}
- return sharedGLCtx.getGL();
+ this.gl = sharedGLCtx.getGL();
}
+
private void makeCurrent(GLContext ctx) {
if( GLContext.CONTEXT_NOT_CURRENT >= ctx.makeCurrent() ) {
throw new GLException("Couldn't make ctx current: "+ctx);
@@ -789,6 +769,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
private void destroySharedGL() {
if( null != sharedGLCtx ) {
+ postNextTextureImpl(gl);
if( sharedGLCtx.isCreated() ) {
// Catch dispose GLExceptions by GLEventListener, just 'print' them
// so we can continue with the destruction.
@@ -870,9 +851,8 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
FramePusherInstanceId++;
synchronized ( this ) {
- if( null != sharedGLCtx ) {
- makeCurrent( sharedGLCtx );
- }
+ makeCurrent( sharedGLCtx );
+ preNextTextureImpl(gl);
isRunning = true;
this.notify(); // wake-up doStart()
}
@@ -880,10 +860,13 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
while( !shallStop ){
if( shallPause ) {
synchronized ( this ) {
+ postNextTextureImpl(gl);
+ sharedGLCtx.release();
while( shallPause && !shallStop ) {
isActive = false;
this.notify(); // wake-up doPause()
try {
+ System.err.println("!!! PAUSE ON"); // FIXME
this.wait(); // wait until resumed
} catch (InterruptedException e) {
if( !shallPause ) {
@@ -891,6 +874,9 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
}
}
+ makeCurrent(sharedGLCtx);
+ preNextTextureImpl(gl);
+ System.err.println("!!! PAUSE OFF"); // FIXME
isActive = true;
this.notify(); // wake-up doResume()
}
@@ -903,15 +889,14 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
nextFrame = videoFramesFree.getBlocking(false /* clearRef */ );
isBlocked = false;
nextFrame.setPTS( TextureFrame.INVALID_PTS ); // mark invalid until processed!
- if( getNextTextureImpl(gl, nextFrame, true) ) {
+ if( getNextTextureImpl(gl, nextFrame, true, false /* issuePreAndPost */) ) {
// gl.glFinish();
gl.glFlush(); // even better: sync object!
if( !videoFramesDecoded.put(nextFrame) ) {
- throw new InternalError("XXX: "+GLMediaPlayerImpl.this);
+ throw new InternalError("XXX: free "+videoFramesFree+", decoded "+videoFramesDecoded+", "+GLMediaPlayerImpl.this);
}
- final TextureFrame _nextFrame = nextFrame;
+ newFrameAvailable(nextFrame, Platform.currentTimeMillis());
nextFrame = null;
- newFrameAvailable(_nextFrame);
}
} catch (InterruptedException e) {
isBlocked = false;
@@ -927,6 +912,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
}
}
+ postNextTextureImpl(gl);
destroySharedGL();
synchronized ( this ) {
isRunning = false;
@@ -975,7 +961,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
if( this.fps != fps ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_FPS;
this.fps = fps;
- this.frame_period = (int) ( 1000f / fps + 0.5f );
+ this.frame_duration = 1000f / (float)fps;
}
if( this.bps_stream != bps_stream || this.bps_video != bps_video || this.bps_audio != bps_audio ) {
event_mask |= GLMediaEventListener.EVENT_CHANGE_BPS;
@@ -1006,7 +992,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected final void attributesUpdated(int event_mask) {
synchronized(eventListenersLock) {
for(Iterator i = eventListeners.iterator(); i.hasNext(); ) {
- i.next().attributesChanges(this, event_mask, System.currentTimeMillis());
+ i.next().attributesChanges(this, event_mask, Platform.currentTimeMillis());
}
}
}
@@ -1017,6 +1003,12 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
destroyFramePusher();
destroyImpl(gl);
removeAllTextureFrames(gl);
+ if( null != videoFramesFree ) {
+ videoFramesFree.clear();
+ }
+ if( null != videoFramesDecoded ) {
+ videoFramesDecoded.clear();
+ }
state = State.Uninitialized;
return state;
}
@@ -1094,25 +1086,28 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
final String loc = ( null != urlConn ) ? urlConn.getURL().toExternalForm() : "" ;
final int freeVideoFrames = null != videoFramesFree ? videoFramesFree.size() : 0;
final int decVideoFrames = null != videoFramesDecoded ? videoFramesDecoded.size() : 0;
- return "GLMediaPlayer["+state+", frames[p "+presentedFrameCount+", d "+decodedFrameCount+", t "+videoFrames+" ("+tt+" s)], "+
+ final int video_scr = video_scr_pts + (int) ( ( Platform.currentTimeMillis() - video_scr_t0 ) * playSpeed );
+ return "GLMediaPlayer["+state+", vSCR "+video_scr+", frames[p "+presentedFrameCount+", d "+decodedFrameCount+", t "+videoFrames+" ("+tt+" s)], "+
"speed "+playSpeed+", "+bps_stream+" bps, "+
"Texture[count "+textureCount+", free "+freeVideoFrames+", dec "+decVideoFrames+", target "+toHexString(textureTarget)+", format "+toHexString(textureFormat)+", type "+toHexString(textureType)+"], "+
- "Video[id "+vid+", <"+vcodec+">, "+width+"x"+height+", "+fps+" fps, "+bps_video+" bps], "+
+ "Video[id "+vid+", <"+vcodec+">, "+width+"x"+height+", "+fps+" fps, "+frame_duration+" fdur, "+bps_video+" bps], "+
"Audio[id "+aid+", <"+acodec+">, "+bps_audio+" bps, "+audioFrames+" frames], uri "+loc+"]";
}
@Override
public final String getPerfString() {
- final int scr_pts = video_scr_pts +
- (int) ( ( System.currentTimeMillis() - video_scr_t0 ) * playSpeed );
- final int d_vpts = video_pts_last - scr_pts;
- return getPerfStringImpl( scr_pts, video_pts_last, d_vpts, getVideoDPTSAvg() );
- }
- private final String getPerfStringImpl(final int scr_pts, final int video_pts, final int d_vpts, final int video_dpts_avg_diff) {
- final float tt = getDuration() / 1000.0f;
- final int audio_scr = (int) ( ( System.currentTimeMillis() - audio_scr_t0 ) * playSpeed );
+ final long currentTimeMillis = Platform.currentTimeMillis();
+ final int video_scr = video_scr_pts + (int) ( ( currentTimeMillis - video_scr_t0 ) * playSpeed );
+ final int d_vpts = video_pts_last - video_scr;
+ final int audio_scr = (int) ( ( currentTimeMillis - audio_scr_t0 ) * playSpeed );
final int audio_pts = getAudioPTSImpl();
final int d_apts = audio_pts - audio_scr;
+ return getPerfStringImpl( video_scr, video_pts_last, d_vpts, audio_scr, audio_pts, d_apts, getVideoDPTSAvg() );
+ }
+ private final String getPerfStringImpl(final int video_scr, final int video_pts, final int d_vpts,
+ final int audio_scr, final int audio_pts, final int d_apts,
+ final int video_dpts_avg_diff) {
+ final float tt = getDuration() / 1000.0f;
final String audioSinkInfo;
final AudioSink audioSink = getAudioSink();
if( null != audioSink ) {
@@ -1122,8 +1117,8 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
}
final int freeVideoFrames = null != videoFramesFree ? videoFramesFree.size() : 0;
final int decVideoFrames = null != videoFramesDecoded ? videoFramesDecoded.size() : 0;
- return state+", frames[p "+presentedFrameCount+", d "+decodedFrameCount+", t "+videoFrames+" ("+tt+" s)], "+
- "speed " + playSpeed+", vSCR "+scr_pts+", vpts "+video_pts+", dSCR["+d_vpts+", avrg "+video_dpts_avg_diff+"], "+
+ return state+", frames[(p "+presentedFrameCount+", d "+decodedFrameCount+") / "+videoFrames+", "+tt+" s], "+
+ "speed " + playSpeed+", dAV "+( d_vpts - d_apts )+", vSCR "+video_scr+", vpts "+video_pts+", dSCR["+d_vpts+", avrg "+video_dpts_avg_diff+"], "+
"aSCR "+audio_scr+", apts "+audio_pts+" ( "+d_apts+" ), "+audioSinkInfo+
", Texture[count "+textureCount+", free "+freeVideoFrames+", dec "+decVideoFrames+"]";
}
@@ -1163,5 +1158,4 @@ public abstract class GLMediaPlayerImpl implements GLMediaPlayer {
protected static final String toHexString(int v) {
return "0x"+Integer.toHexString(v);
}
-
}
\ No newline at end of file
diff --git a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
index 5d70ca33d..ad8587e6b 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/NullGLMediaPlayer.java
@@ -37,6 +37,7 @@ import javax.media.opengl.GLProfile;
import jogamp.opengl.util.av.GLMediaPlayerImpl;
import com.jogamp.common.nio.Buffers;
+import com.jogamp.common.os.Platform;
import com.jogamp.common.util.IOUtil;
import com.jogamp.opengl.util.av.GLMediaPlayer;
import com.jogamp.opengl.util.texture.Texture;
@@ -51,7 +52,7 @@ import com.jogamp.opengl.util.texture.TextureSequence;
public class NullGLMediaPlayer extends GLMediaPlayerImpl {
private TextureData texData = null;
private int pos_ms = 0;
- private int pos_start = 0;
+ private long pos_start = 0;
public NullGLMediaPlayer() {
super();
@@ -64,7 +65,7 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
@Override
protected final boolean playImpl() {
- pos_start = (int)System.currentTimeMillis();
+ pos_start = Platform.currentTimeMillis();
return true;
}
@@ -81,14 +82,14 @@ public class NullGLMediaPlayer extends GLMediaPlayerImpl {
}
@Override
- protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking, boolean issuePreAndPost) {
nextFrame.setPTS( getAudioPTSImpl() );
return true;
}
@Override
protected final int getAudioPTSImpl() {
- pos_ms = (int)System.currentTimeMillis() - pos_start;
+ pos_ms = (int) ( Platform.currentTimeMillis() - pos_start );
validatePos();
return pos_ms;
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
index 3680da1a8..cf864daa2 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java
@@ -55,9 +55,11 @@ import com.jogamp.common.util.RunnableExecutor;
* Tue Feb 28 12:07:53 2012 322537478b63c6bc01e640643550ff539864d790 minor 1 -> 2
*/
class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
+ private static final boolean DEBUG = FFMPEGMediaPlayer.DEBUG || DynamicLibraryBundleInfo.DEBUG;
+
private static final List glueLibNames = new ArrayList(); // none
- private static final int symbolCount = 32;
+ private static final int symbolCount = 38;
private static final String[] symbolNames = {
"avcodec_version",
"avformat_version",
@@ -71,17 +73,20 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
"avcodec_open",
"avcodec_alloc_frame",
"avcodec_default_get_buffer",
- "avcodec_default_release_buffer",
+ "avcodec_default_release_buffer",
+ "av_init_packet",
+ "av_new_packet",
+ "av_destruct_packet",
"av_free_packet",
"avcodec_decode_audio4", // 53.25.0 (opt)
"avcodec_decode_audio3", // 52.23.0
-/* 15 */ "avcodec_decode_video2", // 52.23.0
+/* 18 */ "avcodec_decode_video2", // 52.23.0
// libavutil
"av_pix_fmt_descriptors",
"av_free",
"av_get_bits_per_pixel",
-/* 19 */ "av_samples_get_buffer_size",
+/* 22 */ "av_samples_get_buffer_size",
// libavformat
"avformat_alloc_context",
@@ -93,10 +98,13 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
"av_dump_format",
"av_read_frame",
"av_seek_frame",
+ "avformat_seek_file",
+ "av_read_play",
+ "av_read_pause",
"avformat_network_init", // 53.13.0 (opt)
"avformat_network_deinit", // 53.13.0 (opt)
"avformat_find_stream_info", // 53.3.0 (opt)
-/* 32 */ "av_find_stream_info",
+/* 38 */ "av_find_stream_info",
};
// alternate symbol names
@@ -203,7 +211,7 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
for(int j=0; !ok && j, but has alternative <"+symbolNames[si]+">");
}
}
@@ -212,7 +220,7 @@ class FFMPEGDynamicLibraryBundleInfo implements DynamicLibraryBundleInfo {
System.err.println("Fail: Could not resolve symbol <"+symbolNames[i]+">: not optional, no alternatives.");
return false;
}
- } else if(true || DEBUG ) { // keep it verbose per default for now ..
+ } else if(DEBUG) {
System.err.println("OK: Unresolved optional symbol <"+symbolNames[i]+">");
}
}
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
index dc7ceae39..8998f689a 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java
@@ -156,7 +156,6 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
protected int[] vLinesize = { 0, 0, 0 }; // per plane
protected int[] vTexWidth = { 0, 0, 0 }; // per plane
protected int texWidth, texHeight; // overall (stuffing planes in one texture)
- protected ByteBuffer texCopy;
protected String singleTexComp = "r";
protected GLPixelStorageModes psm;
@@ -174,7 +173,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
if(!available) {
throw new RuntimeException("FFMPEGMediaPlayer not available");
}
- moviePtr = createInstance0(DEBUG);
+ moviePtr = createInstance0( DEBUG_NATIVE );
if(0==moviePtr) {
throw new GLException("Couldn't create FFMPEGInstance");
}
@@ -185,8 +184,6 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
protected final int validateTextureCount(int desiredTextureCount) {
return desiredTextureCount>2 ? Math.max(4, desiredTextureCount) : 2;
}
- @Override
- protected final boolean requiresOffthreadGLCtx() { return true; }
@Override
protected final void destroyImpl(GL gl) {
@@ -320,7 +317,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
case BGRA:
texWidth = vTexWidth[0]; texHeight = height;
break;
- default: // FIXME: Add more planar formats !
+ default: // FIXME: Add more formats !
throw new RuntimeException("Unsupported pixelformat: "+vPixelFmt);
}
@@ -358,7 +355,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
signed = true;
fixedP = true;
break;
- default: // FIXME: Add more planar formats !
+ default: // FIXME: Add more formats !
throw new RuntimeException("Unsupported sampleformat: "+aSampleFmt);
}
avChosenAudioFormat = new AudioDataFormat(AudioDataType.PCM, sampleRate, sampleSize, channels, signed, fixedP, true /* littleEndian */);
@@ -427,7 +424,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
" return vec4(r, g, b, 1);\n"+
"}\n"
;
- default: // FIXME: Add more planar formats !
+ default: // FIXME: Add more formats !
return super.getTextureLookupFragmentShaderImpl();
}
}
@@ -437,7 +434,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
if(0==moviePtr) {
return false;
}
- return true;
+ return play0(moviePtr);
}
@Override
@@ -445,7 +442,7 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
if(0==moviePtr) {
return false;
}
- return true;
+ return pause0(moviePtr);
}
@Override
@@ -457,28 +454,41 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
}
@Override
- protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ protected void preNextTextureImpl(GL gl) {
+ psm.setUnpackAlignment(gl, 1); // RGBA ? 4 : 1
+ }
+
+ @Override
+ protected void postNextTextureImpl(GL gl) {
+ psm.restore(gl);
+ }
+
+ @Override
+ protected final boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking, boolean issuePreAndPost) {
if(0==moviePtr) {
throw new GLException("FFMPEG native instance null");
- }
- psm.setUnpackAlignment(gl, 1); // RGBA ? 4 : 1
- int avPTS = 0;
+ }
+ if( issuePreAndPost ) {
+ preNextTextureImpl(gl);
+ }
+ int vPTS = TextureFrame.INVALID_PTS;
try {
final Texture tex = nextFrame.getTexture();
gl.glActiveTexture(GL.GL_TEXTURE0+getTextureUnit());
tex.enable(gl);
tex.bind(gl);
- /** Try decode up to 10 packets to find one containing video, i.e. vPTS > 0 */
- for(int retry=10; 0 >= avPTS && 0 < retry; retry--) {
- avPTS = readNextPacket0(moviePtr, textureTarget, textureFormat, textureType);
- retry--;
+ /** Try decode up to 10 packets to find one containing video. */
+ for(int i=0; TextureFrame.INVALID_PTS == vPTS && 10 > i; i++) {
+ vPTS = readNextPacket0(moviePtr, textureTarget, textureFormat, textureType);
}
} finally {
- psm.restore(gl);
+ if( issuePreAndPost ) {
+ postNextTextureImpl(gl);
+ }
}
- if( 0 < avPTS ) {
- nextFrame.setPTS(avPTS);
+ if( TextureFrame.INVALID_PTS != vPTS ) {
+ nextFrame.setPTS(vPTS);
return true;
} else {
return false;
@@ -492,6 +502,11 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
}
}
+ private final int getBytesPerMS(int time) {
+ final int bytesPerSample = sinkChosenAudioFormat.sampleSize >>> 3; // /8
+ return time * ( sinkChosenAudioFormat.channelCount * bytesPerSample * ( sinkChosenAudioFormat.sampleRate / 1000 ) );
+ }
+
@Override
protected final boolean syncAVRequired() { return true; }
@@ -522,10 +537,12 @@ public class FFMPEGMediaPlayer extends GLMediaPlayerImpl {
private native Buffer getAudioBuffer0(long moviePtr, int plane);
/**
- * @return resulting current PTS: audio < 0, video > 0, invalid == 0
+ * @return resulting current video PTS, or {@link TextureFrame#INVALID_PTS}
*/
private native int readNextPacket0(long moviePtr, int texTarget, int texFmt, int texType);
+ private native boolean play0(long moviePtr);
+ private native boolean pause0(long moviePtr);
private native int seek0(long moviePtr, int position);
public static enum SampleFormat {
diff --git a/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java b/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
index d03cad28a..c6f31d81e 100644
--- a/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
+++ b/src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
@@ -156,7 +156,7 @@ public class OMXGLMediaPlayer extends EGLMediaPlayerImpl {
}
@Override
- protected boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking) {
+ protected boolean getNextTextureImpl(GL gl, TextureFrame nextFrame, boolean blocking, boolean issuePreAndPost) {
if(0==moviePtr) {
throw new GLException("OMX native instance null");
}
diff --git a/src/jogl/native/libav/ffmpeg_tool.h b/src/jogl/native/libav/ffmpeg_tool.h
index 081e17323..06c3862db 100644
--- a/src/jogl/native/libav/ffmpeg_tool.h
+++ b/src/jogl/native/libav/ffmpeg_tool.h
@@ -68,13 +68,16 @@ typedef GLenum (APIENTRYP PFNGLGETERRORPROC) (void);
/** Sync w/ GLMediaPlayer.STREAM_ID_AUTO */
#define AV_STREAM_ID_AUTO -1
+/** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE 0x80000000 {@value}. Sync w/ TextureFrame.INVALID_PTS */
+#define INVALID_PTS 0x80000000
+
#define AV_HAS_API_REQUEST_CHANNELS(pAV) (AV_VERSION_MAJOR(pAV->avcodecVersion) < 55)
static inline float my_av_q2f(AVRational a){
return a.num / (float) a.den;
}
-static inline int32_t my_av_q2i32(int32_t snum, AVRational a){
- return (snum * a.num) / a.den;
+static inline int32_t my_av_q2i32(int64_t snum, AVRational a){
+ return (int32_t) ( ( snum * (int64_t) a.num ) / (int64_t)a.den );
}
typedef struct {
diff --git a/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c b/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
index bc376cebd..ca0bf9bb9 100644
--- a/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
+++ b/src/jogl/native/libav/jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c
@@ -62,6 +62,9 @@ typedef int (APIENTRYP AVCODEC_OPEN)(AVCodecContext *avctx, AVCodec *codec);
typedef AVFrame *(APIENTRYP AVCODEC_ALLOC_FRAME)(void);
typedef int (APIENTRYP AVCODEC_DEFAULT_GET_BUFFER)(AVCodecContext *s, AVFrame *pic);
typedef void (APIENTRYP AVCODEC_DEFAULT_RELEASE_BUFFER)(AVCodecContext *s, AVFrame *pic);
+typedef void (APIENTRYP AV_INIT_PACKET)(AVPacket *pkt);
+typedef int (APIENTRYP AV_NEW_PACKET)(AVPacket *pkt, int size);
+typedef void (APIENTRYP AV_DESTRUCT_PACKET)(AVPacket *pkt);
typedef void (APIENTRYP AV_FREE_PACKET)(AVPacket *pkt);
typedef int (APIENTRYP AVCODEC_DECODE_AUDIO4)(AVCodecContext *avctx, AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt); // 53.25.0
typedef int (APIENTRYP AVCODEC_DECODE_AUDIO3)(AVCodecContext *avctx, int16_t *samples, int *frame_size_ptr, AVPacket *avpkt); // 52.23.0
@@ -75,6 +78,9 @@ static AVCODEC_OPEN sp_avcodec_open;
static AVCODEC_ALLOC_FRAME sp_avcodec_alloc_frame;
static AVCODEC_DEFAULT_GET_BUFFER sp_avcodec_default_get_buffer;
static AVCODEC_DEFAULT_RELEASE_BUFFER sp_avcodec_default_release_buffer;
+static AV_INIT_PACKET sp_av_init_packet;
+static AV_NEW_PACKET sp_av_new_packet;
+static AV_DESTRUCT_PACKET sp_av_destruct_packet;
static AV_FREE_PACKET sp_av_free_packet;
static AVCODEC_DECODE_AUDIO4 sp_avcodec_decode_audio4; // 53.25.0
static AVCODEC_DECODE_AUDIO3 sp_avcodec_decode_audio3; // 52.23.0
@@ -89,7 +95,7 @@ static const AVPixFmtDescriptor* sp_av_pix_fmt_descriptors;
static AV_FREE sp_av_free;
static AV_GET_BITS_PER_PIXEL sp_av_get_bits_per_pixel;
static AV_SAMPLES_GET_BUFFER_SIZE sp_av_samples_get_buffer_size;
-// count: 19
+// count: 22
// libavformat
typedef AVFormatContext *(APIENTRYP AVFORMAT_ALLOC_CONTEXT)(void);
@@ -101,6 +107,9 @@ typedef int (APIENTRYP AVFORMAT_OPEN_INPUT)(AVFormatContext **ps, const char *fi
typedef void (APIENTRYP AV_DUMP_FORMAT)(AVFormatContext *ic, int index, const char *url, int is_output);
typedef int (APIENTRYP AV_READ_FRAME)(AVFormatContext *s, AVPacket *pkt);
typedef int (APIENTRYP AV_SEEK_FRAME)(AVFormatContext *s, int stream_index, int64_t timestamp, int flags);
+typedef int (APIENTRYP AVFORMAT_SEEK_FILE)(AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags);
+typedef int (APIENTRYP AV_READ_PLAY)(AVFormatContext *s);
+typedef int (APIENTRYP AV_READ_PAUSE)(AVFormatContext *s);
typedef int (APIENTRYP AVFORMAT_NETWORK_INIT)(void); // 53.13.0
typedef int (APIENTRYP AVFORMAT_NETWORK_DEINIT)(void); // 53.13.0
typedef int (APIENTRYP AVFORMAT_FIND_STREAM_INFO)(AVFormatContext *ic, AVDictionary **options); // 53.3.0
@@ -115,13 +124,16 @@ static AVFORMAT_OPEN_INPUT sp_avformat_open_input;
static AV_DUMP_FORMAT sp_av_dump_format;
static AV_READ_FRAME sp_av_read_frame;
static AV_SEEK_FRAME sp_av_seek_frame;
+static AVFORMAT_SEEK_FILE sp_avformat_seek_file;
+static AV_READ_PLAY sp_av_read_play;
+static AV_READ_PAUSE sp_av_read_pause;
static AVFORMAT_NETWORK_INIT sp_avformat_network_init; // 53.13.0
static AVFORMAT_NETWORK_DEINIT sp_avformat_network_deinit; // 53.13.0
static AVFORMAT_FIND_STREAM_INFO sp_avformat_find_stream_info; // 53.3.0
static AV_FIND_STREAM_INFO sp_av_find_stream_info;
-// count: 32
+// count: 38
-#define SYMBOL_COUNT 32
+#define SYMBOL_COUNT 38
JNIEXPORT jboolean JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGDynamicLibraryBundleInfo_initSymbols0
(JNIEnv *env, jclass clazz, jobject jSymbols, jint count)
@@ -152,17 +164,20 @@ JNIEXPORT jboolean JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGDynamicLibraryB
sp_avcodec_alloc_frame = (AVCODEC_ALLOC_FRAME) (intptr_t) symbols[i++];
sp_avcodec_default_get_buffer = (AVCODEC_DEFAULT_GET_BUFFER) (intptr_t) symbols[i++];
sp_avcodec_default_release_buffer = (AVCODEC_DEFAULT_RELEASE_BUFFER) (intptr_t) symbols[i++];
+ sp_av_init_packet = (AV_INIT_PACKET) (intptr_t) symbols[i++];
+ sp_av_new_packet = (AV_NEW_PACKET) (intptr_t) symbols[i++];
+ sp_av_destruct_packet = (AV_DESTRUCT_PACKET) (intptr_t) symbols[i++];
sp_av_free_packet = (AV_FREE_PACKET) (intptr_t) symbols[i++];
sp_avcodec_decode_audio4 = (AVCODEC_DECODE_AUDIO4) (intptr_t) symbols[i++];
sp_avcodec_decode_audio3 = (AVCODEC_DECODE_AUDIO3) (intptr_t) symbols[i++];
sp_avcodec_decode_video2 = (AVCODEC_DECODE_VIDEO2) (intptr_t) symbols[i++];
- // count: 15
+ // count: 18
sp_av_pix_fmt_descriptors = (const AVPixFmtDescriptor*) (intptr_t) symbols[i++];
sp_av_free = (AV_FREE) (intptr_t) symbols[i++];
sp_av_get_bits_per_pixel = (AV_GET_BITS_PER_PIXEL) (intptr_t) symbols[i++];
sp_av_samples_get_buffer_size = (AV_SAMPLES_GET_BUFFER_SIZE) (intptr_t) symbols[i++];
- // count: 19
+ // count: 22
sp_avformat_alloc_context = (AVFORMAT_ALLOC_CONTEXT) (intptr_t) symbols[i++];;
sp_avformat_free_context = (AVFORMAT_FREE_CONTEXT) (intptr_t) symbols[i++];
@@ -173,11 +188,14 @@ JNIEXPORT jboolean JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGDynamicLibraryB
sp_av_dump_format = (AV_DUMP_FORMAT) (intptr_t) symbols[i++];
sp_av_read_frame = (AV_READ_FRAME) (intptr_t) symbols[i++];
sp_av_seek_frame = (AV_SEEK_FRAME) (intptr_t) symbols[i++];
+ sp_avformat_seek_file = (AVFORMAT_SEEK_FILE) (intptr_t) symbols[i++];
+ sp_av_read_play = (AV_READ_PLAY) (intptr_t) symbols[i++];
+ sp_av_read_pause = (AV_READ_PAUSE) (intptr_t) symbols[i++];
sp_avformat_network_init = (AVFORMAT_NETWORK_INIT) (intptr_t) symbols[i++];
sp_avformat_network_deinit = (AVFORMAT_NETWORK_DEINIT) (intptr_t) symbols[i++];
sp_avformat_find_stream_info = (AVFORMAT_FIND_STREAM_INFO) (intptr_t) symbols[i++];
sp_av_find_stream_info = (AV_FIND_STREAM_INFO) (intptr_t) symbols[i++];
- // count: 32
+ // count: 38
(*env)->ReleasePrimitiveArrayCritical(env, jSymbols, symbols, 0);
@@ -502,7 +520,7 @@ JNIEXPORT void JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_setStre
// Customize ..
// pAV->pACodecCtx->thread_count=2;
// pAV->pACodecCtx->thread_type=FF_THREAD_FRAME|FF_THREAD_SLICE; // Decode more than one frame at once
- pAV->pACodecCtx->thread_count=1;
+ pAV->pACodecCtx->thread_count=0;
pAV->pACodecCtx->thread_type=0;
pAV->pACodecCtx->workaround_bugs=FF_BUG_AUTODETECT;
pAV->pACodecCtx->skip_frame=AVDISCARD_DEFAULT;
@@ -579,7 +597,7 @@ JNIEXPORT void JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_setStre
// Customize ..
// pAV->pVCodecCtx->thread_count=2;
// pAV->pVCodecCtx->thread_type=FF_THREAD_FRAME|FF_THREAD_SLICE; // Decode more than one frame at once
- pAV->pVCodecCtx->thread_count=1;
+ pAV->pVCodecCtx->thread_count=0;
pAV->pVCodecCtx->thread_type=0;
pAV->pVCodecCtx->workaround_bugs=FF_BUG_AUTODETECT;
pAV->pVCodecCtx->skip_frame=AVDISCARD_DEFAULT;
@@ -684,10 +702,12 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
FFMPEGToolBasicAV_t *pAV = (FFMPEGToolBasicAV_t *)((void *)((intptr_t)ptr));
AVPacket packet;
- int frameFinished;
- jint resPTS = 0; // resulting current PTS: audio < 0, video > 0, invalid == 0
+ int frameDecoded;
+ jint resPTS = INVALID_PTS;
- if(sp_av_read_frame(pAV->pFormatCtx, &packet)>=0) {
+ sp_av_init_packet(&packet);
+
+ if( sp_av_read_frame(pAV->pFormatCtx, &packet) >= 0 ) {
if(packet.stream_index==pAV->aid) {
// Decode audio frame
if(NULL == pAV->pAFrames) { // no audio registered
@@ -704,10 +724,10 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
break;
}
if(HAS_FUNC(sp_avcodec_decode_audio4)) {
- len1 = sp_avcodec_decode_audio4(pAV->pACodecCtx, pAFrameCurrent, &frameFinished, &packet);
+ len1 = sp_avcodec_decode_audio4(pAV->pACodecCtx, pAFrameCurrent, &frameDecoded, &packet);
} else {
#if 0
- len1 = sp_avcodec_decode_audio3(pAV->pACodecCtx, int16_t *samples, int *frame_size_ptr, &frameFinished, &packet);
+ len1 = sp_avcodec_decode_audio3(pAV->pACodecCtx, int16_t *samples, int *frame_size_ptr, &frameDecoded, &packet);
#endif
JoglCommon_throwNewRuntimeException(env, "Unimplemented: FFMPEGMediaPlayer sp_avcodec_decode_audio3 fallback");
return 0;
@@ -720,7 +740,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
packet.data += len1;
packet.size -= len1;
- if (!frameFinished) {
+ if (!frameDecoded) {
// stop sending empty packets if the decoder is finished
if (!packet.data && pAV->pACodecCtx->codec->capabilities & CODEC_CAP_DELAY) {
flush_complete = 1;
@@ -745,7 +765,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
const int64_t pkt_pts = pAFrameCurrent->pkt_pts;
int aptsMode;
if( 0 == frameCount && AV_NOPTS_VALUE != pkt_pts ) { // 1st frame only, discard invalid PTS ..
- pAV->aPTS = (pkt_pts * (int64_t) 1000 * (int64_t) time_base.num) / (int64_t) time_base.den ;
+ pAV->aPTS = my_av_q2i32( pkt_pts * 1000, time_base);
aptsMode = 0;
} else { // subsequent frames or invalid PTS ..
const int32_t bytesPerSample = 2; // av_get_bytes_per_sample( pAV->pACodecCtx->sample_fmt );
@@ -753,14 +773,15 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
aptsMode = 1;
}
if( pAV->verbose ) {
- fprintf(stderr, "A pts %d [pkt_pts %ld, pkt_dts %ld], dataSize %d, f# %d, pts-mode %d\n",
- pAV->aPTS, pkt_pts, pAFrameCurrent->pkt_dts, data_size, frameCount, aptsMode);
+ int32_t aDTS = my_av_q2i32( pAFrameCurrent->pkt_dts * 1000, time_base);
+
+ fprintf(stderr, "A pts %d [pkt_pts %ld], dts %d [pkt_dts %ld], dataSize %d, f# %d, pts-mode %d\n",
+ pAV->aPTS, pkt_pts, aDTS, pAFrameCurrent->pkt_dts, data_size, frameCount, aptsMode);
}
if( NULL != env ) {
jobject jSampleData = (*env)->NewDirectByteBuffer(env, pAFrameCurrent->data[0], data_size);
(*env)->CallVoidMethod(env, instance, jni_mid_pushSound, jSampleData, data_size, pAV->aPTS);
}
- resPTS = pAV->aPTS * -1; // Audio Frame!
}
} else if(packet.stream_index==pAV->vid) {
// Decode video frame
@@ -775,7 +796,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
if (flush_complete) {
break;
}
- len1 = sp_avcodec_decode_video2(pAV->pVCodecCtx, pAV->pVFrame, &frameFinished, &packet);
+ len1 = sp_avcodec_decode_video2(pAV->pVCodecCtx, pAV->pVFrame, &frameDecoded, &packet);
if (len1 < 0) {
// if error, we skip the frame
packet.size = 0;
@@ -784,7 +805,7 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
packet.data += len1;
packet.size -= len1;
- if (!frameFinished) {
+ if (!frameDecoded) {
// stop sending empty packets if the decoder is finished
if (!packet.data && pAV->pVCodecCtx->codec->capabilities & CODEC_CAP_DELAY) {
flush_complete = 1;
@@ -796,15 +817,25 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
const AVRational time_base = pAV->pVStream->time_base;
const int64_t pkt_pts = pAV->pVFrame->pkt_pts;
if(AV_NOPTS_VALUE != pkt_pts) { // discard invalid PTS ..
- int32_t vPTS2 = (pAV->pVFrame->pkt_dts * (int64_t) 1000 * (int64_t) time_base.num) / (int64_t) time_base.den ;
- pAV->vPTS = (pkt_pts * (int64_t) 1000 * (int64_t) time_base.num) / (int64_t) time_base.den ;
+ pAV->vPTS = my_av_q2i32( pkt_pts * 1000, time_base);
if( pAV->verbose ) {
- fprintf(stderr, "V pts %d [pkt_pts %ld], pts2 %d [pkt_dts %ld]\n", pAV->vPTS, pkt_pts, vPTS2, pAV->pVFrame->pkt_dts);
- }
- } else {
- if( pAV->verbose ) {
- fprintf(stderr, "V pts ?? [pkt_pts %ld], pts2 ?? [pkt_dts %ld]\n", pkt_pts, pAV->pVFrame->pkt_dts);
+ int32_t vDTS = my_av_q2i32( pAV->pVFrame->pkt_dts * 1000, time_base);
+
+ double frame_delay_d = av_q2d(pAV->pVCodecCtx->time_base);
+ double frame_repeat_d = pAV->pVFrame->repeat_pict * (frame_delay_d * 0.5);
+
+ int32_t frame_delay_i = my_av_q2i32(1000, pAV->pVCodecCtx->time_base);
+ int32_t frame_repeat_i = pAV->pVFrame->repeat_pict * (frame_delay_i / 2);
+
+ const char * warn = frame_repeat_i > 0 ? "REPEAT" : "NORMAL" ;
+
+ fprintf(stderr, "V pts %d [pkt_pts %ld], dts %d [pkt_dts %ld], time d(%lf s + r %lf = %lf s), i(%d ms + r %d = %d ms) - %s - f# %d\n",
+ pAV->vPTS, pkt_pts, vDTS, pAV->pVFrame->pkt_dts,
+ frame_delay_d, frame_repeat_d, (frame_delay_d + frame_repeat_d),
+ frame_delay_i, frame_repeat_i, (frame_delay_i + frame_repeat_i), warn, frameCount);
}
+ } else if( pAV->verbose ) {
+ fprintf(stderr, "V pts ?? [pkt_pts %ld], pts2 ?? [pkt_dts %ld], f# %d\n", pkt_pts, pAV->pVFrame->pkt_dts, frameCount);
}
resPTS = pAV->vPTS; // Video Frame!
@@ -846,6 +877,31 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_readNex
return resPTS;
}
+JNIEXPORT jboolean JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_play0
+ (JNIEnv *env, jobject instance, jlong ptr)
+{
+ FFMPEGToolBasicAV_t *pAV = (FFMPEGToolBasicAV_t *)((void *)((intptr_t)ptr));
+ int res = sp_av_read_play(pAV->pFormatCtx);
+ if ( 0 != res && -ENOSYS != res ) { // Ignore ENOSYS (not impl.)
+ fprintf(stderr, "PLAY: err %d 0x%X\n", res, res);
+ return JNI_FALSE;
+ } else {
+ return JNI_TRUE;
+ }
+}
+JNIEXPORT jboolean JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_pause0
+ (JNIEnv *env, jobject instance, jlong ptr)
+{
+ FFMPEGToolBasicAV_t *pAV = (FFMPEGToolBasicAV_t *)((void *)((intptr_t)ptr));
+ int res = sp_av_read_pause(pAV->pFormatCtx);
+ if ( 0 != res && -ENOSYS != res ) { // Ignore ENOSYS (not impl.)
+ fprintf(stderr, "PAUSE: err %d 0x%X\n", res, res);
+ return JNI_FALSE;
+ } else {
+ return JNI_TRUE;
+ }
+}
+
JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_seek0
(JNIEnv *env, jobject instance, jlong ptr, jint pos1)
{
@@ -854,14 +910,14 @@ JNIEXPORT jint JNICALL Java_jogamp_opengl_util_av_impl_FFMPEGMediaPlayer_seek0
int64_t pts0 = pAV->pVFrame->pkt_pts;
int64_t pts1 = (int64_t) (pos1 * (int64_t) pAV->pVStream->time_base.den)
/ (1000 * (int64_t) pAV->pVStream->time_base.num);
+
int flags = 0;
if(pos1 < pos0) {
flags |= AVSEEK_FLAG_BACKWARD;
}
fprintf(stderr, "SEEK: pre : u %ld, p %ld -> u %ld, p %ld\n", pos0, pts0, pos1, pts1);
sp_av_seek_frame(pAV->pFormatCtx, pAV->vid, pts1, flags);
- pAV->vPTS = (int64_t) (pAV->pVFrame->pkt_pts * (int64_t) 1000 * (int64_t) pAV->pVStream->time_base.num)
- / (int64_t) pAV->pVStream->time_base.den;
+ pAV->vPTS = my_av_q2i32( pAV->pVFrame->pkt_pts * 1000, pAV->pVStream->time_base);
fprintf(stderr, "SEEK: post : u %ld, p %ld\n", pAV->vPTS, pAV->pVFrame->pkt_pts);
return pAV->vPTS;
}
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
index fbbd77260..b673a9d2a 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube.java
@@ -33,9 +33,7 @@
package com.jogamp.opengl.test.junit.jogl.demos.es2.av;
-import java.io.BufferedReader;
import java.io.IOException;
-import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
@@ -102,20 +100,16 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
int pts0 = mPlayer.getVideoPTS();
int pts1 = 0;
switch(e.getKeyCode()) {
- case KeyEvent.VK_3:
case KeyEvent.VK_RIGHT: pts1 = pts0 + 1000; break;
- case KeyEvent.VK_4:
case KeyEvent.VK_UP: pts1 = pts0 + 10000; break;
- case KeyEvent.VK_2:
+ case KeyEvent.VK_PAGE_UP: pts1 = pts0 + 30000; break;
case KeyEvent.VK_LEFT: pts1 = pts0 - 1000; break;
- case KeyEvent.VK_1:
case KeyEvent.VK_DOWN: pts1 = pts0 - 10000; break;
+ case KeyEvent.VK_PAGE_DOWN: pts1 = pts0 - 30000; break;
case KeyEvent.VK_ESCAPE:
- case KeyEvent.VK_DELETE:
+ case KeyEvent.VK_HOME:
case KeyEvent.VK_BACK_SPACE: {
- mPlayer.setPlaySpeed(1.0f);
mPlayer.seek(0);
- mPlayer.play();
break;
}
case KeyEvent.VK_SPACE: {
@@ -126,6 +120,9 @@ public class MovieCube implements GLEventListener, GLMediaEventListener {
}
break;
}
+ case KeyEvent.VK_MULTIPLY:
+ mPlayer.setPlaySpeed(1.0f);
+ break;
case KeyEvent.VK_SUBTRACT: {
float playSpeed = mPlayer.getPlaySpeed();
if( e.isShiftDown() ) {
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
index f5490d19a..af9454464 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieSimple.java
@@ -164,20 +164,16 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
int pts0 = mPlayer.getVideoPTS();
int pts1 = 0;
switch(e.getKeyCode()) {
- case KeyEvent.VK_3:
case KeyEvent.VK_RIGHT: pts1 = pts0 + 1000; break;
- case KeyEvent.VK_4:
case KeyEvent.VK_UP: pts1 = pts0 + 10000; break;
- case KeyEvent.VK_2:
+ case KeyEvent.VK_PAGE_UP: pts1 = pts0 + 30000; break;
case KeyEvent.VK_LEFT: pts1 = pts0 - 1000; break;
- case KeyEvent.VK_1:
case KeyEvent.VK_DOWN: pts1 = pts0 - 10000; break;
+ case KeyEvent.VK_PAGE_DOWN: pts1 = pts0 - 30000; break;
case KeyEvent.VK_ESCAPE:
- case KeyEvent.VK_DELETE:
+ case KeyEvent.VK_HOME:
case KeyEvent.VK_BACK_SPACE: {
- mPlayer.setPlaySpeed(1.0f);
mPlayer.seek(0);
- mPlayer.play();
break;
}
case KeyEvent.VK_SPACE: {
@@ -188,6 +184,9 @@ public class MovieSimple implements GLEventListener, GLMediaEventListener {
}
break;
}
+ case KeyEvent.VK_MULTIPLY:
+ mPlayer.setPlaySpeed(1.0f);
+ break;
case KeyEvent.VK_SUBTRACT: {
float playSpeed = mPlayer.getPlaySpeed();
if( e.isShiftDown() ) {
--
cgit v1.2.3
From 4dc4a32720e7b176e6811c0eaa8ddc060e1468da Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Fri, 23 Aug 2013 00:39:30 +0200
Subject: TextureSequence: Add END_OF_STREAM_PTS, remove 'blocking' from
getNextTexture(..), may blocks .. or not, depending on implementation and
state.
---
.../jogamp/opengl/util/texture/TextureSequence.java | 21 ++++++++++++---------
.../junit/jogl/demos/TextureSequenceDemo01.java | 2 +-
.../jogl/demos/es2/TextureSequenceCubeES2.java | 2 +-
3 files changed, 14 insertions(+), 11 deletions(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
index 05fda99ae..e13e5ff13 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java
@@ -110,8 +110,11 @@ public interface TextureSequence {
* to associated related data.
*/
public static class TextureFrame {
- /** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE 0x80000000 {@value}. */
- public static final int INVALID_PTS = 0x80000000 ; // == -2147483648 == Integer.MIN_VALUE;
+ /** Constant marking an invalid PTS, i.e. Integer.MIN_VALUE == 0x80000000 == {@value}. Sync w/ native code. */
+ public static final int INVALID_PTS = 0x80000000;
+
+ /** Constant marking the end of the stream PTS, i.e. Integer.MIN_VALUE - 1 == 0x7FFFFFFF == {@value}. Sync w/ native code. */
+ public static final int END_OF_STREAM_PTS = 0x7FFFFFFF;
public TextureFrame(Texture t) {
texture = t;
@@ -130,7 +133,7 @@ public interface TextureSequence {
public final void setDuration(int duration) { this.duration = duration; }
public String toString() {
- return "TextureFrame[pts " + pts + " ms, l " + duration + " ms, "+ texture + "]";
+ return "TextureFrame[pts " + pts + " ms, l " + duration + " ms, texID "+ texture.getTextureObject() + "]";
}
protected final Texture texture;
protected int pts;
@@ -141,7 +144,7 @@ public interface TextureSequence {
/**
* Signaling listeners that a new {@link TextureFrame} is available.
*
- * User shall utilize {@link TextureSequence#getNextTexture(GL, boolean)} to dequeue it to maintain
+ * User shall utilize {@link TextureSequence#getNextTexture(GL)} to dequeue it to maintain
* a consistent queue.
*
* @param ts the event source
@@ -163,7 +166,7 @@ public interface TextureSequence {
*
* In case the instance is just initialized, it shall return a TextureFrame
* object with valid attributes. The texture content may be undefined
- * until the first call of {@link #getNextTexture(GL, boolean)}.
+ * until the first call of {@link #getNextTexture(GL)}.
*
* Not blocking.
*
@@ -174,16 +177,16 @@ public interface TextureSequence {
/**
* Returns the next texture to be rendered.
*
- * Implementation shall block until next frame is available if blocking is true,
- * otherwise it shall return the last frame in case a new frame is not available.
+ * Implementation shall return the next frame if available, may block if a next frame may arrive soon.
+ * Otherwise implementation shall return the last frame.
*
*
- * Shall return null in case no frame is available.
+ * Shall return null in case no next or last frame is available.
*
*
* @throws IllegalStateException if instance is not initialized
*/
- public TextureFrame getNextTexture(GL gl, boolean blocking) throws IllegalStateException ;
+ public TextureFrame getNextTexture(GL gl) throws IllegalStateException ;
/**
* In case a shader extension is required, based on the implementation
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/TextureSequenceDemo01.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/TextureSequenceDemo01.java
index 6fd47e63f..ab3899a7b 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/TextureSequenceDemo01.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/TextureSequenceDemo01.java
@@ -77,7 +77,7 @@ public class TextureSequenceDemo01 implements TextureSequence {
}
@Override
- public TextureSequence.TextureFrame getNextTexture(GL gl, boolean blocking) throws IllegalStateException {
+ public TextureSequence.TextureFrame getNextTexture(GL gl) throws IllegalStateException {
return frame;
}
diff --git a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
index 556d17992..4172a2c20 100644
--- a/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
+++ b/src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java
@@ -370,7 +370,7 @@ public class TextureSequenceCubeES2 implements GLEventListener {
interleavedVBO.enableBuffer(gl, true);
Texture tex = null;
if(null!=texSeq) {
- final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl, true);
+ final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl);
if(null != texFrame) {
tex = texFrame.getTexture();
gl.glActiveTexture(GL.GL_TEXTURE0+texSeq.getTextureUnit());
--
cgit v1.2.3
From d0e01cb5c0ec3e48b8a9b9b79a7795b214c6e3ea Mon Sep 17 00:00:00 2001
From: Sven Gothel
Date: Sat, 24 Aug 2013 17:56:49 +0200
Subject: GLMediaPlayer Multithreaded Decoding: GLMediaPlayer* (Part-6) - DONE
Multithreaded decoding and API should be considered stable by now,
minor changes may apply if Android/OMX impl. requires it.
We still need to solve TODO's as listed below, copied from 474ce65081ecd452215bc07ab866666cb11ca8b1.
+++
- *TextureFrame OO changes:
- TextureFrame extends TimeFrameI
- GLMediaPlayerImpl*
- Adapt to Ringbuffer changes of GlueGen commit f9f881e59c78e3036cb3f956bc97cfc3197f620d
- Fix impl. method's API doc
- getNextTextureImpl(..) returns video PTS
- Fix audio-only playback
- frame dropping shall only happen if:
- previous frame has not been dropped
- frame is too later
- one decoded frame is already available
- Don't block for decoder anymore:
- nextFrame = "videoFramesDecoded.getBlocking() -> videoFramesDecoded.get()";
No 'next decoded frame avail' only could mean:
- slow decoding/hardware
- slow transport
hence we shall not block rendering.
- Add DEBUG output if using last frame
- Add integer property 'jogl.debug.GLMediaPlayer.StreamWorker.delay' in milliseconds
to simulate slow decoding, i.e. delay is added in StreamWorker after decoding
before pushing new frame to Ringbuffer.
- FFMPEGMediaPlayer:
- audioFrameLimitWithVideo 128 -> 64
- audioFrameLimitAudioOnly 128 -> 32
- uses AudioSink's 'enqueueData(int pts, ByteBuffer bytes, int byteCount)'
- fixes for audio-only playback
+++
Working Tests: MovieSimple and MovieCube
TODO-1: Fix
- Android
- OMXGLMediaPlayer
TODO-2:
- Fix issue where async audio frames arrive much later than 1st video frame, i.e. around 300ms.
- Default TextureCount .. maybe 3 ?
- Adding Audio synchronization ?
- Find 'truth' about correlation of audio and video PTS values,
currently, we assume both to be unrelated ?
---
make/scripts/tests.sh | 5 +-
.../com/jogamp/opengl/util/av/GLMediaPlayer.java | 12 +-
.../opengl/util/texture/TextureSequence.java | 28 +--
.../android/av/AndroidGLMediaPlayerAPI14.java | 9 +-
.../jogamp/opengl/util/av/GLMediaPlayerImpl.java | 253 +++++++++++++--------
.../jogamp/opengl/util/av/NullGLMediaPlayer.java | 41 ++--
.../opengl/util/av/impl/FFMPEGMediaPlayer.java | 43 ++--
.../opengl/util/av/impl/OMXGLMediaPlayer.java | 5 +-
src/jogl/native/libav/ffmpeg_tool.h | 4 +-
.../jogamp_opengl_util_av_impl_FFMPEGMediaPlayer.c | 44 ++--
.../opengl/test/android/MovieCubeActivity0.java | 4 +-
.../opengl/test/android/MovieSimpleActivity0.java | 4 +-
.../opengl/test/android/MovieSimpleActivity1.java | 4 +-
.../jogl/demos/es2/TextureSequenceCubeES2.java | 35 +--
.../test/junit/jogl/demos/es2/av/MovieCube.java | 24 +-
.../test/junit/jogl/demos/es2/av/MovieSimple.java | 74 +++---
16 files changed, 322 insertions(+), 267 deletions(-)
(limited to 'src/jogl/classes/com/jogamp/opengl/util/texture/TextureSequence.java')
diff --git a/make/scripts/tests.sh b/make/scripts/tests.sh
index 6475f06b4..c7207bbcf 100644
--- a/make/scripts/tests.sh
+++ b/make/scripts/tests.sh
@@ -134,6 +134,7 @@ function jrun() {
#D_ARGS="-Djogl.debug.GLMediaPlayer -Djogl.debug.AudioSink"
#D_ARGS="-Djogl.debug.GLMediaPlayer -Djogl.debug.GLMediaPlayer.Native"
#D_ARGS="-Djogl.debug.GLMediaPlayer"
+ #D_ARGS="-Djogl.debug.GLMediaPlayer.StreamWorker.delay=25 -Djogl.debug.GLMediaPlayer"
#D_ARGS="-Djogl.debug.GLMediaPlayer.Native"
#D_ARGS="-Djogl.debug.AudioSink"
#D_ARGS="-Djogl.debug.GLArrayData"
@@ -318,8 +319,8 @@ function testawtswt() {
# av demos
#
#testnoawt jogamp.opengl.openal.av.ALDummyUsage $*
-testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
-#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
+#testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube $*
+testnoawt com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple $*
#
# core/newt (testnoawt and testawt)
diff --git a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
index 726eddb01..02fbd721c 100644
--- a/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
+++ b/src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java
@@ -35,6 +35,7 @@ import javax.media.opengl.GLException;
import jogamp.opengl.Debug;
import com.jogamp.opengl.util.texture.TextureSequence;
+import com.jogamp.opengl.util.TimeFrameI;
/**
* GLMediaPlayer interface specifies a {@link TextureSequence} state machine
@@ -116,14 +117,11 @@ import com.jogamp.opengl.util.texture.TextureSequence;
* to be properly considered by {@link GLMediaPlayerFactory#create(ClassLoader, String)}
* and {@link GLMediaPlayerFactory#createDefault()}.
*
+ *
- * Variable type, value range and dimension has been chosen to suit embedded CPUs
- * and characteristics of audio and video streaming.
- * Milliseconds of type integer with a maximum value of {@link Integer#MAX_VALUE}
- * will allow tracking time up 2,147,483.647 seconds or
- * 24 days 20 hours 31 minutes and 23 seconds.
- * Milliseconds granularity is also more than enough to deal with A-V synchronization,
- * where the threshold usually lies within 22ms.
+ *
+ * Timestamp type and value range has been chosen to suit embedded CPUs
+ * and characteristics of audio and video streaming. See {@link TimeFrameI}.
*