Windows Vista introduced helpful video and audio Digital Signal Processors (DSPs) in DMO form-factor, which however do not work smoothly with DMO Wrapper Filter and thus cannot be directly used in DirectShow.

There perhaps was no intent in first place to extend DirectShow functionality with these new components, and no effort was put into providing this mode of operation, however as long as the new classes are DMOs, it is still possible to tune them up to work in DirectShow pipeline.

This sample code/application provides a code snippet on how Video Resizer DSP can be used in DirectShow. There were earlier some discussions on MSDN Forums and this complements the guidelines with code.

The idea is the following:

• CoCreateInstance the DSP as DMO and add it to DMO Wrapper Filter
• Use IWMResizerProps::SetFullCropRegion to initialize the DSP
• Connect input pin
• Set output type via IMediaObject::SetOutputType
• IGraphBuilder::ConnectDirect output pin

The sample application takes a video file (note that for the brevity of the sample, not all files will be supported, there is an assumption that files are decoded into VIDEOINFOHEADER media type, and we limit color spaces to 32-bit RGB only to avoid problems on the way in this tiny sample).

The application takes file path (I recommend .WMV) creates DirectShow pipeline, adds Sample Grabber filter to force color space to be 32-bit RGB, adds resizer and sets it up to double video height, but not width, and plays the video.

The application’s filter graph looks like this:

And the video window is stretched twice in height:

A binary [Win32] and Visual C++ .NET 2010 source code [Trac, Subversion] are available from SVN; the important part goes here.

Although HRESULT codes are so common and structure is simple and well known, and even Visual Studio helps decoding the values nowadays, looking up for code takes some effort: hexadecimal value, searching through SDK headers or online, overlapped regions of codes in FACILITY_ITF etc.

MSDN describes the codes in the following sections:

• Structure of COM Error Codes
• Codes in FACILITY_ITF
• Error Handling Strategies

Now is there an answer to the challenge of easy decoding without routinely looking up Windows SDK every time? A helper system tray application makes a nice try. I used to use HR Plus (HR+) application for some time, but at some point even its small bugs started being annoying. Additionally, this one has a priority to decoding DirectShow, Windows Media and Media Foundation error codes.

So having the app started, copy HRESULT or plain system error code such as ERROR_ACCESS_DENIED (5) into clipboard, decimal or hexadecimal, and have it popped up. A click on the balloon googles for code details.

Let us see what is in there. We are using FormatMessage API to look for code message in the following order of preference: DirectShow, Windows Media, Media Foundation, Windows Sockets, WinTHTP, WinInet, System. If the code is found – and this really covers a huge number of codes, there might be additional lookup for the code identifier.

CString sTitle = _T("System"), sMessage, sIdentifier;
if(IsQuartzResult(nResult, &sMessage))
{
    sTitle = _T("DirectShow");
    sIdentifier = LookupQuartzIdentifier(nResult);
} else if(IsWmResult(nResult, &sMessage))
    sTitle = _T("Windows Media");
else if(IsMfResult(nResult, &sMessage))
{
    sTitle = _T("Media Foundation");
    sIdentifier = LookupMfIdentifier(nResult);
} else if(IsWs2Result(nResult, &sMessage))
    sTitle = _T("Sockets");
else if(IsWinHttpResult(nResult, &sMessage))
    sTitle = _T("WinHTTP");
else if(IsWinInetResult(nResult, &sMessage))
    sTitle = _T("WinInet");
else 
{
    sMessage = AtlFormatSystemMessage(nResult);
    sIdentifier = LookupSystemIdentifier(nResult);
    if(sIdentifier.IsEmpty())
        sIdentifier = LookupHresultSystemIdentifier(nResult);
}

If the code is found, it goes up into balloon notification.

The application decodes 6000+ HRESULT codes into human friendly state.

Download links:

• Partial Visual C++ 2012 source code: Trac, Subversion
• Binaries: Win32 – ShowHresult.exe

persiflage@stackoverflow asks if there is a chance to use DirectShow Spy see who sent an EC_ERRORABORT notification, which filter exactly. Let us see first why there is no way to find this out, and then we will see what we can do.

DirectShow Filter Graph Manager accepts events from filters via its IMediaEventSink interface. The conversation taking place around event notifications is like this:

• Filter: Hey, Graph Manager! Can I call you IMediaEventSink?
• Manager: Yes, you can.
• F: I notify you on EC_ERRORABORT event, here is HRESULT that I have: VFW_E_SOMETHING.
• M: OK.

Filter graph manager (FGM) does not ask “Who’s taking?”. It does not need to know, it accepts information anonymously. Can a non-filter post an event? Absolutely, FGM does not have to care. This is simple, but when a question raised who posted the event, there is no answer for it – there was no such information in first place.

The good news through is that a developer does not need one hundred percent precision. The source of the event is important to understand which of the filters aborted streaming, and any information is helpful. Spy impersonates the whole FGM and as such it is capable of covering IMediaEventSink interface as well, in order to trace calls to the log file, and, even more helpful, trace call stack of the call which brought specific event in.

With the call stack information at the time of event notification, the filter of interest can be identified pretty precisely. Especially, having debug symbols available, so that Spy could provide symbols for the code locations on stack.

For instance, let us looks at Windows SDK AMCap Sample which previews video and uses video renderer, and hence has EC_VIDEO_SIZE_CHANGED event involved (just an example, spy from now on traces EC_ERRORABORT call stack only). Once this event reaches FGM, the call stack logged is:

FilterGraphSpy.h(850): CSpyT<class CSpy,&struct _GUID const CLSID_FilterGraph>::Notify: nEventCode EC_VIDEO_SIZE_CHANGED (0x0A), nParameter1 0x00F00140, nParameter2 0x00000000
  DirectShowSpy!6a3ba3b4 CSpyT<CSpy,&CLSID_FilterGraph>::Notify (+ 337) [d:\projects\alax.info\repository-public\directshowspy\filtergraphspy.h, 859] (+ 13) @6a3a0000
  quartz!6a243188 CBaseFilter::NotifyEvent (+ 46) @6a220000
  quartz!6a3394f6 CBaseControlVideo::OnVideoSizeChange (+ 56) @6a220000
  quartz!6a2a2f9a CRenderer::CompleteConnect (+ 175) @6a220000
  quartz!6a337668 CRendererInputPin::CompleteConnect (+ 25) @6a220000
  quartz!6a23a470 CBasePin::ReceiveConnection (+ 213) @6a220000
  quartz!6a2a3741 CVideoInputPin::ReceiveConnection (+ 92) @6a220000
  ksproxy!65e94dc0 CBasePin::AttemptConnection (+ 84) @65e70000
  ksproxy!65e94e81 CBasePin::TryMediaTypes (+ 104) @65e70000
  ksproxy!65e94f68 CBasePin::AgreeMediaType (+ 115) @65e70000
  ksproxy!65e966a0 CBasePin::Connect (+ 100) @65e70000
  ksproxy!65e7d711 CKsOutputPin::Connect (+ 381) @65e70000
  quartz!6a23252e CFilterGraph::ConnectDirectInternal (+ 233) @6a220000
  quartz!6a23847c CFilterGraph::ConnectRecursively (+ 44) @6a220000
  quartz!6a238d09 CFilterGraph::ConnectInternal (+ 331) @6a220000
  quartz!6a238c22 CFilterGraph::Connect (+ 23) @6a220000
  DirectShowSpy!6a3b8686 CSpyT<CSpy,&CLSID_FilterGraph>::Connect (+ 881) [d:\projects\alax.info\repository-public\directshowspy\filtergraphspy.h, 672] (+ 0) @6a3a0000
  quartz!6a2322f0 CEnumMediaTypes::Release (+ 39) @6a220000
  qcap!6a6c7a31 CBuilder2_2::DoesCategoryAndTypeMatch (+ 408) @6a6b0000
  qcap!6a6b3424 _GUID_00000000_0000_0000_0000_000000000000 (+ 4) @6a6b0000
  qcap!6a6cb9cb CBuilder2_2::RenderStream (+ 5294) @6a6b0000
  AMCap!01009723 @01000000
  AMCap!010041be @01000000
  AMCap!01005e27 @01000000
  AMCap!0100611c @01000000
  AMCap!01007600 @01000000
  AMCap!010076ba @01000000
  AMCap!0100a90d @01000000

It does not take a rocket scientist to see that event is posted by video renderer hosted by quartz.dll, which was a part of pin connection handling, where a pin of ksproxy’s filter – which has to be WDM Video Capture Filter – was connected to video renderer input pin.

DirectShow Spy started logging new items:

• COM interface calls on filter graph IMediaEvent, IMediaEventEx, IMediaEventSink interfaces
• Call staclk on IMediaEventSink::Notify call, with EC_ERRORABORT code (other codes are logged without call stack to reduce hook overhead and avoid logging stuff for no reason)

Download links:

• Binaries: Win32 – DirectShowSpy.dll (1.0.0.647), x64 – DirectShowSpy.dll (1.0.0.646)
• Instructions: original post

As libx264 has so many presets and tunes, I was curious how they all related one to another when it comes to encode video info H.264. I was more interested in single pass encoding for live video, so the measurements are respectively for this mode of operation with encoder running in CRF (constant rate factor, X264_RC_CRF).

So I took Robotica_1080.wmv HD video in 1440×1080 resolution and batch-transcoded into H.264 using libx264 (build 128) in various modes operation:

• Presets: “ultrafast”, “superfast”, “veryfast”, “faster”, “fast”, “medium”, “slow”, “slower”, “veryslow”
• Tunes: “film”, “animation”, “grain”, “stillimage”, “psnr”, “ssim”, “fastdecode”, “zerolatency”, “touhou”
• CRFs: 14, 17, 20, 23, 26

It is worth mentioning that libx264 does EXCELLENT job in transcoding in terms of performance. Transcoding operation was a DirectShow graph of the following topology:

Some measurements are obviously not quite accurate because not only encoding time counts, WMV decoding time counts also etc. Still this should give a good idea how modes stand side by side one with another.

For every transcoding run I have the following values (Excel spreadsheet attached below):

• Processor Time: number of processor-milliseconds spent on the transcoding; I was measuring in 8 core system, so with 100% load processor time could be up to eight times higher than Elapsed Time (below) provided that all cores were used in full
• Elapsed Time: milliseconds spent on the transcoding; regardless of how many actual cores were in use, because original clip is 20 seconds long everything below that is faster than realtime processing
• Output File Size: size of resulting MP4 video only file, some headers count as well however it is obviously mostly payload data; for a 20 seconds clip, 20 MB is 8 mbit/s bitrate

Another derivative value is:

• Processor Time/Elapsed Time: which shows fullness of use of multicore system; some modes are clearly not using all available cores, while other do

Let us start watching pictures.

Average Elapsed Time for Preset/Tune (covers runs with different rate factors) shows that slow+ modes take exponentially more time for encoding. psnr and ssim tunes do transcoding slightly faster, while zerolatency tune is the most expensive.

ultrafast and superfast presets produced significantly larger files, about 2x as large as other presets.

Once again exponential scale of Elapsed Time, and similar Processor Time chart:

It is worth mentioning that fastest presets are not using all CPU cores. Apart from being faster on their own, they leave some CPU time for other processing which can be useful for live encoding applications, and those processing multiple streams at once.

And finally detailed file size dependency from preset and CRF rate. As we already discovered, ultrafast and superfast produce larger stream, while output of other modes not so much differ (within a few percent, mostly on the slowest end). A step in rate factor of three gives about 0.7x decrease in amount of produces bytes.

More fun charts can be obtained from the attached .XLS file.

Download links:

• Raw Data

This publishes a small update to IP Video Source tool to expose M-JPEG/JPEG video streams as DriectShow video devices. The utility now accepts https:// URL scheme and pulls video over secure connections. Not every camera has support for HTTPS, however many do including those from UDP Technology, Axis etc.

The update also includes other minor fixes including more reasonable memory allocation, and hotfix for D-Link DCS-930L model, which – to mention – is an excellent indication of quality of camera firmware nowadays (it has nothing to do with D-Link, most of vendors do things like this):

#pragma region D-Link DCS-930L Fix
// NOTE: This is just ridiculous, the camera sends boundary with space without quotes AND misuses -- prefix
if(sValue.Compare(L"multipart/x-mixed-replace;boundary=--video boundary--") == 0)
    sValue = L"multipart/x-mixed-replace;boundary=\"video boundary--\"";
#pragma endregion

Download links

• Version: 1.0.2
• Binaries: Win32 – IpVideoSourceSetup-Win32.msi, x64 – IpVideoSourceSetup-x64.msi
• License: This software is free to use

A new release of Skype is out there, version 6.0.0.126:

• Metro look on the main (roster) window – good job!
• Broken compatibility with software (virtual) video capture devices – good job!
• Broken compatibility with some of hardware capture devices – good job!

While stripping virtual video devices in favor of true cameras, to avoid issues, might be intentional (though it would still be hard to justify anyway), the update also screwed the operation with a true hardware WDM driver backed PCI video capture board… Nice – with a dozen of cameras around the table I no longer have a single one compatible with Skype.

A little bit of hackery brings virtual DirectShow sources back to life, including those created by IP Video Source, however the issue is clearly Skype’s: I have no intention to work it around, let’s hope Skype guys are given some piece of mind to fix that soon.

Be aware and DO NOT UPGRADE if operation of virtual video source in Skype is is essential!

NOTE: There is an open issue on Skype’s tracker for the virtual camera compatibility.

Because DirectShow Spy is often a troubleshooting tool, one of its use scenarios is its being a drop-in module to quickly install on a system of interest in order to connect to graphs for troubleshooting purposes, such as to check topology and media types.

Its installation requires COM registration, and over time it changed gradually from simple to more and more complex step. Why? In Windows XP one had to open command prompt and regsvr32. With Vista’s UAC one needs a prompt, with privilege elevation, which opens typically in wrong directory, then UAC prompt. A relatively easy step became annoying multi-step operation. Then proxy/stub classes were moved into Windows SDK DLL…

Things are getting back to be easier with DirectShow Spy. It is given a special property sheet right there withing the DLL, to take care of all the important things:

• checks registration status
• buttons to Unregister/Register
• per-user registration (not recommended though due to system wide class hooking)
• automatically takes care of UAC prompt
• place Windows SDK proppage.dll into the same directory near spy, and additional property page will help you to register this additional dependency

To invoke registration UI, start:

rundll32 DirectShowSpy.dll,DoRegistrationPropertySheetModal

from command line, or just have a .BAT file ready to do it for you.

Download links

• Binaries: Win32 – DirectShowSpy.dll (1.0.0.807), x64 – DirectShowSpy.dll (1.0.0.806)
• License: This software is free to use
• Instructions: original post

As mentioned before, new Skype 6 broke compatibility with all virtual DirectShow devices out there. Just oops, nothing works any longer if only it is not a full driver exposing virtual device through WDM.

Since quite some people are interested in details (Skype 6 Virtual Camera issues, Skype Client for Windows – SCW-3881 – Virtual cameras no longer work with Skype 6), here goes a bit of relevant information on the problem.

New Skype still uses DirectShow as API to interface capture devices and to discover such it enumerates them the way all decent applications do: Enumerating Devices and Filters. Presumably, there was a good reason to touch the video capture code and from moderately terrible it started being even worse. Skype guys decided that if a video device does not have a DevicePath property then it is not good enough for them.

The “DevicePath” property is not a human-readable string, but is guaranteed to be unique for each video capture device on the system. You can use this property to distinguish between two or more instances of the same model of device.

The documentation is unfortunately confusing in this part and perhaps developers thought they were doing the right thing, nevertheless multiple discussions online suggest a different approach.

Unfortunately, DevicePath is not a mandatory property. And it does not exist in virtual devices, and you cannot use it to distinguish between them. And Skype started – supposedly – skipping devices without it. And this is why all virtual devices were left overboard.

Virtual devices are not that rare. It might not only be a gate into IP camera, instead it might be a device that splits exclusive use type video camera device between applications, or something that adds an overlay onto captured image. That is, the whole class of devices failed to work with Skype from there. Including, of course, IP Video Source which is a virtual DirectShow device with the backend network connection into an IP camera or video encoder.

Adding missing DevicePath is the key to fix the problem and give Skype what it thinks is a must. Let’s hope we have a fix from Skype soon and there is no need in working things around any longer, but if someone needs a quick solution – it also exists (apart from rewriting everything into kernel mode driver, and other hackery).

Since the properties an application reads from IPropertyBag interface (which is in turn obtained from IMoniker when enumerating devices) are [partially] backed by system registry, it is sufficient to add “DevicePath” value into specific key of the registry and make the fake property available for those devices that don’t have kernel path.

The registry key is located under HKLM, SOFTWARE\Classes\CLSID\{860BB310-5D01-11d0-BD3B-00A0C911CE86}\Instance (note it’s SOFTWARE\Classes\Wow6432Node\CLSID... in 64-bit OS for 32-bit app space), where every subkey corresponds to a registered device (find yours there). Note that CLSID above is actually CLSID_VideoInputDeviceCategory.

Adding new value there will create a fake property on the property bag and improve compatibility with such picky software as Skype. Note that the value will be destroyed with re-registration of the device filter, and will have to be created once again.

A small issue appears to be affecting DirectShow applications with DirectShow Spy installed. As underlying COM base is non-standard, the spy implements a few hacks to run smoothly and to keep reference counting correct in particular. Under certain conditions, DirectShow-enabled ActiveX control hosted by Internet Explorer seems to be unable to put its graph onto Running Object Table (ROT). What happened next is that Spy assumed ROT operation to succeed, and compensated reference counting, which under bad assumption could cause E_UNEXPECTED error while creating a filter graph. This updated fixes the issue.

Another small improvement is that similarly to SDK proppage.dll, Spy registration UI also assists in registering another DLL – evrprop.dll, should it be there near the spy module.

Download links

• Binaries: Win32 – DirectShowSpy.dll (1.0.0.829), x64 – DirectShowSpy.dll (1.0.0.828)
• License: This software is free to use
• Instructions: original post

Someone discovered the utility while browsing my public repository (the app prompts to post data back to the website, and the anonymous user accepted the offer and posted the report from this unpublished application), so I have to drop a few lines about the tool.

The idea is basically straightforward: live capture involves attaching time stamps to media samples, and there is a chance that the time stamps slide away causing unwanted effects on captured clip. The application captures video and audio simultaneously and tracks media sample time stamps, and compares them against system clock as well. Having it simply run for a few minutes one can see how the capture is doing and if any of the timings drift away. Being stopped it puts report onto clipboard and optionally posts it back to me online (no actually specific intent about this data, however if you want to share data for a device that does drift away, you are to only click once to send me the details).

The output is on clipboard in tab-separated values (TSV) format:

Computer Name   PSI
Windows Version 6.1.7601 Service Pack 1
Video Device    Conexant's BtPCI Capture    @device:pnp:\\?\pci#ven_109e&dev_036e&subsys_18511851&rev_02#4&39c3dd91&0&08f0#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
Audio Device    Stereo Mix (Realtek High Defini @device:cm:{33D9A762-90C8-11D0-BD43-00A0C911CE86}\Stereo Mix (Realtek High Defini

System Time Video Sample Count  Video Sample Time   Relative Video Sample Time  Audio Sample Count  Audio Sample Time   Relative Audio Sample Time
30439   907 30381   -57 304 30291   -147

Or you might prefer pasting it onto Excel:

By the way, this is also an easy way to ensure devices are operational and check effective video frame rate.

Download links:

• Partial Visual C++ .NET 2010 source code: Trac, Subversion
• Binaries: Win32 – CaptureClock.exe

A typical task for DirectShow Spy is to get installed onto, make the target application filter graphs available for inspection and… then get inspected with some additional tool. The last step assumes there is an additional application ready to pick the graphs and an interactive user attaches to the graphs for visual inspection and troubleshooting.

There are great tools to access DirectShow filter graphs via Running Object Table, GraphEdit, GraphStudio and GraphStudioNext – to name a few. However quite so often the task in question is to simply grab information about the filter graph topology and take it away. DirectShow Spy now offers this capability:

The property sheet lists the available ROT filter graphs in a developer-friendly way:

• sorted in reverse chronological order to easily pick the one of interest
• Check button updates filter count and state of the selected graphs
• Copy button puts a user friendly filter graph description onto clipboard, for one or several selected filter graphs:
  • information is formatted using Markdown formatting (both readable as raw text and even better via HTML conversion; a sample filter graph information is shown below)
  • the text mentions graph state, filters, connections and media types

Clipboard integration makes it really easy to pass the information through remote access applications.

More to that, the “graph to text” conversion is available programmatically. C# code snippet (full C# project is here) as simple as the following makes Markdown filter graph information available to the application for logging or otherwise troubleshooting purposes:

[STAThread]
static void Main(string[] args)
{
    IFilterGraph2 graph = new FilterGraph() as IFilterGraph2;
    graph.RenderFile(@"E:\Media\GoPro 2010 Highlights - You in HD - 1920x1080.mp4", "");
    FilterGraphHelper helper = new FilterGraphHelper();
    helper.FilterGraph = graph;
    Console.WriteLine(helper.Text);
}

The new UI is available through exported function DoFilterGraphListPropertySheetModal.

Download links

• Binaries: Win32 – DirectShowSpy-Win32.dll (1.0.0.974), x64 – DirectShowSpy-x64.dll (1.0.0.973)
• Shortcuts to Exported Functions: Helper .BAT files
• License: This software is free to use
• Installation Instructions: Original post

An example of filter graph description is shown below:

One of the recent updates delivered via automatic Windows Update is breaking video decoding for Windows Media Video streams (four character code WMV3). It’s hard to trace things back, however the most likely issue is the update of WMVDECOD.DLL file which is dated 6 May 2013 delivered with MS13-057: Description of the security update for Windows Media Format Runtime 9 and 9.5 (wmvdmod.dll), and for Windows Media Player 11 and 12: July 9, 2013, and it could be one or more versions before as well. The breaking update also affects Windows XP systems.

The problem is affecting both Win32 and x64 systems, and Windows XP and later, however it might be affecting them differently. While in Windows 7 the top half of the video is damaged, in Windows XP the output is totally black. There might be other factors affecting the output though.

The problem is only taking place when WM Video decoder is set to decode into 32-bit RGB format. This limits the impact to application forcing output to RGB for grabbing purposes, or use post processing in RGB domain. Decoding into YUV and 24-bit RGB formats looks correct.

The problem is easy to repeat and it is only necessary to force the feed to decode into 32-bit RGB, for instance using Sample Grabber Filter:

The workaround is not to avoid decoding into 32-bit RGB. In particular, in Windows Vista one can decode into something else and then convert to 32-bit RGB using Color Converter DSP.

Links

• Source Code: Trac/Subversion
• Windows Media Video 9 decoder (WMV3) incorrectly decodes video into 32-bit RGB format on MS Connect
• WMV Decoder Regression due to hotfix on MSDN Forums

This is not really obvious: Windows Media Player refuses to open a Windows Media (ASF) file with an undescriptive error message: “Windows Media Player encountered a problem while playing the file”.

The problem, however, is that the file is actually good, more or less. The file plays well in DirectShow and Media Foundation APIs. There is a unusual thing, of course, that the file contains two audio tracks. The tracks are mutually exclusive as they should be (exclusion of language type – MFASFMutexType_Language).

href=”http://alax.info/blog/wp-content/uploads/2015/04/Image003.png”>

It appears that Windows Media Player does a strict checking of language strings, and expects them to be RFC 1766 compatible. They are free style tags in this file:

Not a valid language string? OK, no playback then.

The tool appears to be unmentioned here, and this is to fix the problem.

DirectShowCaptureCapabilities application enumerates video and audio capture devices and lists their typical DirectShow properties, specifically:

• Moniker names (including USB identification)
• Pins and property pages
• Supported interfaces
• Formats and capabilities available through IAMStreamConfig interface
• Video and audio devices

The utility allows to save output and post it over Internet, what some users already did and here is capabilities of some hardware.

Some of the files might be useful to provide sample data for AM_MEDIA_TYPE structures for typical YUY2, UYVY, MJPG, H264 formats.

Download:

• DirectShowCaptureCapabilities-Win32.exe (32-bit)
• DirectShowCaptureCapabilities-x64.exe (64-bit)

A few direct links to user-submitted output:

• (Aveo) USB WebCam.md
• GC310, OV2680, USB2.0 PC CAMERA.md
• Integrated Camera.md
• Integrated Webcam, Dazzle Video Capture USB Video Device.md
• Integrated Webcam, Izone Full HD 1080p Web Camera.md
• Integrated Webcam, LEO_UVCDEMO_TEST.md
• Integrated Webcam, VDP Source.md
• Lenovo Monitor Integrated Camera, WebCamGhost, HD TV CAM, VisioForge Network Source Video, VisioForge Virtual Camera.md
• Logitech HD Pro Webcam C920 (2).md
• Logitech HD Pro Webcam C920, HP Truevision HD.md
• Logitech HD Pro Webcam C920, XSplitBroadcaster.md
• Logitech HD Pro Webcam C920.md
• Logitech HD Webcam C270.md
• Logitech HD Webcam C310, Integrated Webcam.md
• Logitech QuickCam S5500, Virtual.md
• Logitech Webcam C930e.md
• Microsoft LifeCam HD-5001.md
• Microsoft LifeCam VX-1000.md
• Noname USB Webcam.md
• OV5640, LG HD WebCam.md
• Sensoray 2253 Decode, Sensoray 2253 Capture.md
• USB HD Webcam, USB2.0 Camera.md
• USB Video Device, Integrated Webcam.md
• USB2.0 HD UVC WebCam.md
• USB2.0 UVC HQ WebCam.md

Overview

DirectShow filters pass media samples (portions of data) through graphs and details of how the streaming happens exactly is important for debugging and troubleshooting DirectShow graphs and development. A developer needs clear understanding of parts of streaming process, importance of which increases with multiple streams, threads, parallelization, cut-off times, multiple graphs working simultaneously.

Details of streaming is typically hidden from top level control over the graph where the application is controlling state of the process overall, and then filters are on their own sending data through.

DirectShowSpy provides API to filters to register media samples as well as other details of streaming process, including comments and application defined emphasis (highlighting), it stores the traces and provides UI to review and export the traces for analysis and troubleshooting.

A similar tracing is implemented by GraphStudioNext Analyzer Filter.

DirectShowSpy trace is different in several ways:

1. DirectShowSpy is a drop-in module and adds troubleshooting capabilities to already built and existing application, including that it is suitable for temporary troubleshooting in production environment
   • DirectShowSpy offers tracing for filter which are private and not registered globally
   • DirectShowSpy tracing better reproduces production application environment
2. DirectShowSpy allows supplementary application defined comments, which are registered chronologically along with media samples tracing
   • it is possible to trace not only at filter boundaries/granularity, but also internal events and steps
3. DirectShowSpy combines tracing from multiple graphs, multiple processes and presents them in a single log

DirectShowSpy media sample trace is a sort of log capability, which is implemented with small overhead. The traces reside in RAM, backed by paging file and are automatically released with release and destruction of filter graph. The important exception is however the media sample tracing UI of DirectShowSpy. When UI is active, each (a) manual refresh of the view, and (b) each destruction of filter graph in analyzed process makes UI add a reference to trace data and data lifetime is extended up to closing of DirectShowSpy UI.

The important feature of behavior mentioned above is that media tracing data outlives, or might outlive the processes that host filter graphs. As long as UI is active, processes including terminated expose media sample trace for interactive review.

Basically the feature is to review a streaming session details obtained from the filters which registered the respective events. For example, this filter graph

has two filters, MPEG-4 demultiplexer and multiplexer, which register streaming events. Because the trace is chronological, it in particular allows seeings how “Stuff” filter is doing the processing: threads, timings. If “Stuff” filter registers its own events, the picture becomes more complete.

Using

To leverage media sample traces, a filter developer obtains ISpy interface from filter graph (which succeeds when DirectShowSpy is registered and hooks between application and DirectShow API) and creates a IMediaSampleTrace interface using ISpy::CreateMediaSampleTrace call. An example of such integration is shows in a fork of GDCL MPEG-4 filters here, in DemuxOutputPin::Active method.

It does not matter whether filter and pins are sharing IMediaSampleTrace pointers. Each CreateMediaSampleTrace creates a new trace object, which is thread safe on its own, and data is anyway combined on UI from all sources of tracing.

With no DirectShowSpy registered, QueryInterface for ISpy fails and this is the only expense/overhead of integration of media sample tracing in production code.

A developer is typically interested in registering the following events:

• Segments starts, media sample receives and deliveries, end of stream events; DirectShowSpy introduces respective methods in IMediaSampleTrace interface: RegisterNewSegment, RegisterMediaSample, RegisterEndOfStream
• Application defined comments, using RegisterComment method

All methods automatically track process and thread information associated with the source of the event. Other parameters include:

• filter interface
• abstract stream name, which is a string value and can be anything; typically it makes sense to use pin name/type or it can be pin name with appended stage of processing if developer wants to track processing steps as they happen in the filter; UI offers filtering capability for stream values, and separate column in exported text so that filter could be applied in spreadsheet software such as Excel when reviewing the log
• user defined comment and highlighting option

RegisterMediaSample methods can be used with anything associated with a media sample, not exactly one event per processing call. The method logs media sample data (it takes AM_SAMPLE2_PROPERTIES pointer as byte array pointer) and makes it available for review with its flags and other data.

Comments can be anything and are to hold supplementary information for events happening in certain relation to streaming:

An application can automatically highlight the log entries to draw attention to certain events. For example, if data is streamed out of order and the filter registers the event with highlighting, the entry immediately drawing attention on UI review. Then interactive user can change the highlighting interactively as well:

The media trace data can be conveniently filtered right in DirectShowSpy UI, which is invoked by DoMediaSampleTracePropertySheetModal exported function, or copied to clipboard or saved as file in tab-separated values format. The file can be opened with Microsoft Excel for further review.

Limitations

• there is a global limit on in-memory trace storage; there is no specific size in samples (it’s 8 MB for global registry of data here) and the storage is large enough to hold streaming of a movie with multiple tracks, however once in a while it is possible to hit the limit and there is no automatic recycling of data: the data is released with last copy of UI closed and graphs released in context of their respective processes
• traces are visible from the same session only, in particular processes with elevated privileges are “visible” by similarly started DirectShowSpy UI and vice versa
• 32-bit process traces are visible from 32-bit DirectShowSpy UI, and the same applies to 64 bits; technically it is possible to share that but it is not implemented

Download links

• Binaries:
  • 32-bit: DirectShowSpy-Win32.dll (1.0.0.1876)
  • 64-bit: DirectShowSpy-x64.dll (1.0.0.1875)
• Shortcuts to Exported Functions: Helper .BAT files
• License: This software is free to use
• Installation Instructions: Original post

Additional stuff

A fork of GDCL MPEG-4 filters is uploaded to GitHub, which in particular has integration with media sample tracing and includes pre-built binaries, 32- and 64-bit versions.

As mentioned recently in DirectShowSpy media sample trace update, I uploaded a fork of MPEG-4 filters developed by Geraint Davies, which includes a few updates made over time. They are worth mentioning in a separate post:

• the projects received type libraries and it is easier to reference filters now by #importing the type library into your project, with class and interface definitions
• added support for seeking to key frame, positioning loop without thread re-creation, ability to query for all frame times in the file including frame flags – all together this makes a base for playback with good scrubbing performance and user experience
• fixed a few bugs (resulting file might be not playable in WMP, seeking esp. when scrubbing results in media samples set off by small amount of time, maybe other)
• ability to affect multiplexer memory allocator and its properties
• support for raw video formats, so that MP4 container could be used as a temporary storage for raw captured video (it’s a good choice because the container format is not bloated and has no file size constraints unlike e.g. AVI implementations)
• ability to write a secondary helper temporary file to keep important data, which lets one fix broken capture sessions; an incomplete MP4 file is typically a piece of garbage but with this file it can be restored up to crash point (think of multi-hour recordings) – this needs to be elaborated and some time in future there could possibly be a tool that does the job
• last but not least, DirectShowSpy integration; the code is easy to remove or disable because it is put inside #if defined(ALAXINFODIRECTSHOWSPY_AVAILABLE) sections; integration serves both as example of how to leverage DirectShowSpy media sample traces and also to provide pre-built filters with enabled tracing

Source

• Welcome to Github: gdcl.co.uk-mpeg4 repository

An interesting find about CLSID_FilterGraphNoThread version of DirectShow fitler graph implementation is that its WaitForCompletion method available through IMediaEvent and IMediaEventEx interfaces is not implemented: the immediately returned value is E_NOTIMPL.

CLSID_FilterGraphNoThread itself is not well documented and is rather a spin off the baseline implementation (which is – my guess would be – was introduced at some later stage and was necessary for windowless filter graphs running in worker threads; perhaps Windows Media Player team requested this at some point of WMP development):

… creates the Filter Graph Manager on the application’s thread. If you use this CLSID, the thread that calls CoCreateInstance must have a message loop that dispatches messages; otherwise, deadlocks can occur. Also, before the application thread exits, it must release the Filter Graph Manager and all graph objects (such as filters, pins, reference clocks, and so forth).

The filter graph does not have its internal worker window and is unable to wait for completion in its usual manner. Hence the error. Those who are using CLSID_FilterGraphNoThread version of the filter graphs are supposed to handle EC_COMPLETE events to detect the completion.

I suppose it is not intended and not even somehow foreseen behavior, but who uses CLSID_FilterGraphNoThread anyway except Windows Media Player and those who can cope with the problem.