When starting to look at various real time video processing software like Isadora (VJ Tool), or openFrameWorks (C++ effects library), I found that I was unable to connect my Panasonic HPX 170 camera and see video in real time. I was always getting errors when connecting the camera, or when starting capture. I knew it could work because Adobe OnLocation and an application called WinDV would render a real time capture to the screen. So I figured there must be a variety of APIs involved here. Thus started my odyssey into figuring out how all this works.
To see if I could make something work myself, I started by looking at Microsoft Media Foundation. With code from that API, I would always get "Invalid Argument" errors. Even when performing testing with the SDK's TopoEdit tool, the same errors occurred. I abandoned that API. If it would have worked, it would have been simple to obtain real time video captures.
Since Isadora is a closed source application, all I could do was submit a bug report and see what happens. I did get a reply back from the author several days later, but nothing substantive has developed yet.
With the WinDV application, source-code is available. I seem to recall that that application used the ISampleGrabber interface. But this interface is deprecated in current DirectShow API's, and the sample code wouldn't easily compile on Windows 7.
openFrameWorks is open source, and thus I was able to follow through the code to see where my problems reside. openFramework's video capture API (ofVideoDevice, ofVideoGrabber ...) uses a third party linked library called videoInput to perform the operation. I had to track down the code to another repository in order to obtain the code and review it. That library also used ISampleGrabber , but in a manner which wouldn't work with the Panasonic drivers. I seem to recall it was trying to use Preview pin semantics, which I subsequently found out is not used by the Panasonic decoding library.
I then downloaded the Windows 7 SDK and Windows 7.1 SDK to see what was available. The SDK has a DvApp sample which compiles and works. It uses the default renderer to render the video to screen.
My requirement is to be able to capture the video, perform further translations, and then use openFramework rendering tools. Thus, to use a modern API and capabilities, I need to create my own DirectShow Filter. Roman Ginzburg has an example of building a filter at
DirectShow Filters Development Part 1: Video Rendering with Direct2D.
When building that DirectShow Filter example, I found out a few things:
- The Panasonic decoder filter only comes in a 32 bit flavour. The decoder pack at
Main Concept DVCPro HD Plugins may have both 32 and 64 bit drivers.
- When building filters, it is an easier task if the SDK samples in ...\Samples\multimedia\directshow\baseclasses are built as a library. And the library needs to be built with 'shared library' attributes. I was unable to regsvr32 the library with 'static library' attributes.
- The article directshow filter registration on x64 contains a few interesting key points. Some of which are listed here.
- "DllEntryPoint@12" has to be changed to "DllEntryPoint" when using x64 mode.
- On an x64 machine, in 32 bit mode, use: c:\windows\syswow64\regsvr32.exe
- On an x64 machine, in 64 bit mode, use: c:\windows\system32\regsvr32.exe
- Something for the future to try based upon filter profiling: Roman's filter will convert YUV to RGB. It would be interesting to know if the DirectShow 'MSYUV Color Space Converter Codec' is any faster or efficient. More filters are found at DirectShow Filters.
I am going to have to give VideoMan Library a try. If it works, I don't have to do everything from scratch. Some time later ..... nope, the examples recognize the the camera, but emit errors when trying to render. Back to the drawing board. But they have lots of interesting use scenarios. I will have to come back and look at code to see what is useful once I have the basic capture in place.
As a side note, for developing, Infognition has an Enhanced GraphEdit application. Also, a starting point for tips and techniques is The March Hare's web site.
Roman's sample filter worked well. The online MSDN documents for DirectShow were helpful for filling in the gaps. I didn't use it, and it might be similar to the MSDN docs, but there is an online .pdf describing Writing DirectShow Filters .