Sunday, March 9. 2014

Interactive Digital Media

Over the last few weeks, I've been delving into the world of Interactive Digital Media. This has included investigation into interactive environments, programming tools, modelling suites, supporting hardware, algorithms, plus a host of related and unrelated alley-ways.

Isadora by Troikatronix: "The award-winning, real-time media manipulation software to create stunningly interactive visuals, sounds, and environments." I have downloaded the demo tool. At this point, I rank this as one of the more capable, flexible mutlmedia design tools. Multiple sources, multiple methods for mixing, multiple projections. Very flexible methods of inter-connecting modules. Two issues I am currently working through: a) program crash during activating live video capture from a Firewire connected video camera, b) I'd like a more interactive approach to creating projection surfaces in the output window. An artist using Isadora: Graham Thorne
MWM: unique 3D projection mapping tool purely GPU based to maximize performances. It takes a bit to get familiar with the user interface. But once figured out, it is quite powerful. If the programmability of Isadora was matched with this, would be cool.
Modul8: Mac OS Real Time Video Mixing
Pure Data: is an open source visual programming language. Pd enables musicians, visual artists, performers, researchers, and developers to create software graphically, without writing lines of code. Pd is used to process and generate sound, video, 2D/3D graphics, and interface sensors, input devices, and MIDI.
OpenNI 2 Virtual Webcam v0.9.5 - DirectShow filter for OpenNI: "access output of any OpenNI compatible device using your favorite messaging, video calling application and any application that use DirectShow to access media devices or objects, including Skype, Live Messenger, Google Plus, ManyCAM, GraphEdit, any flash based video chat, etc". Suggested for use with Isadora for Kinect integration.
OpenNI: an open source SDK used for the development of 3D sensing middleware libraries and applications. Closed after April 23, 2014.
OpenCV: open source computer vision and machine learning software library
ArcheOS: An Archeological Operating System. The page references some interesting visual tools for point-clouds, 3d reconstruction, etc.
Arc-Team: References Python Photogrammetry Toolbox and ArcheOS.
How to make 3d scan with pictures and the PPT GUI
Point Cloud Library: standalone, large scale, open project for 2D/3D image and point cloud processing.
OpenFrameWorks: wiki
MeshLab is an open source, portable, and extensible system for the processing and editing of unstructured 3D triangular meshes. For use with point clouds.
Open Source Photogrammetry
VisualSFM: A Visual Structure From Motion System
KEY Photogrammetry: Open Source Alternatives to 123D Catch: A start place to sites on Photogrammetry.
Wikipedia on Photogrammetry
Open Source Photogrammetry: Ditching 123D Catch: Detailed blog article on using VisualSFM and other tools to develop point clouds.
Poisson mapping for texturing background
CMPMVS - Multi-View Reconstruction Software: Pairs with VisualSFM.
PhotoGrammetry Forums: Everything Related To Photogrammetry.
AnthSpace Analytical Cartography Lab: Photogrammetry, Kite Aerial Photography.
VLFeat.org: open source library implements popular computer vision algorithms including HOG, SIFT, MSER, k-means, hierarchical k-means, agglomerative information bottleneck, SLIC superpixels, and quick shift.
Real Time Video Capture and Benchmarking: is a universal Windows application that can be used with games using DirectX or OpenGL graphic technology. I came across this when figuring out how to get video and images from one application to another. Mac has Syphon, but Windows doesn't.
Vadim Smakhtin: Visual Arts, Motion Graphics - ideas for visuals. Sample Effects Code.
Music Motion Noise: Blog relating to visual media. Take a look at Noise: their set of forums. libpd is mentioned.
LPMT: Simple tool for Projection Mapping. Based upon OpenFrameworks Library. Extensive feature list. Has source code available. Might be good starting point for adding customizations, such as timelines, DMX, live video capture.
LightJams: a unique lighting control software making it easy to react to live events from music, midi, videos and more. Incredible interactivity and automation now possible. Video is integrated. Also DMX.
Amazing Windows VJ Tools: Various visual effects samples, references, ... Good place for ideas. Uses LightJams and Isadora.
Wyphon: Something like Syphon on Windows. But I think there are other ways to do this. Hopefully I have the reference further down here where it is possible to grab an OpenGL handle, and grab the image. Which I think Fraps does. Once you have the handle, then the image can be sent between apps or across the network with OpenFrameworks.
Multipurpose VJ Toolkit: a hybrid graphical/textual programming environment for easy prototyping and development. It is designed to facilitate the handling of large media environments with physical interfaces, real-time motion graphics, audio and video that can interact with many users simultaneously.
freeglut: alternative to the OpenGL Utility Toolkit (GLUT) library. GLUT was originally written by Mark Kilgard to support the sample programs in the second edition OpenGL 'RedBook'. Since then, GLUT has been used in a wide variety of practical applications because it is simple, widely available and highly portable. Many tools use OpenGL for the visual side of things. If I were to do something from the ground up in OpenGL, this might be a native wrapper.
wxGLCanvas: But then again, I have been using wxWidgets for a bunch of stuff, so I could wrap all my OpenGL customizations in wxWidgets window.
Learning Modern 3D Graphics Programming: An online tutorial for getting started with OpenGL. Very good for understanding the interior workings.
Whorld: visualizer for sacred geometry. Ideas for VJ visualizations. Which is a FreeFrame plugin.
Away3D: real time 3D engine for the Flash Platform. Search for Code Orchestra for LiveCoding this environment.
Develop Kinect: community based site focusing on development and programming for the Microsoft Kinect 3D depth sensor
Kinect For Windows Blog
PortAudio: cross-platform, open-source, audio I/O library
PyAudio: Python bindings for PortAudio, the cross-platform audio I/O library. Building PyAudio on Windows 7x64 with MSVC toolchaines. PyAudio with PortAudio with ASIO support.
SoundAnalyse: Python package to analyze sound chunks from microphone. In conjunction with Blender Game Engine.
FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video.
BigFug: Video Mapping, FreeFrame, FFGL, Video Streaming, Audio Processing tools by Alex May
FreeFrame: provides a framework for developing video effects plugins and hosts on Windows, Linux and Mac OSX
Stereographic Depth Mapping on an FPGA
Open Source FPGA Stereo Vision Core: detailed blog entry on how to perform depth mapping from stereo images in an FPGA
How to Detect and Track Object With OpenCV
gPhoto2: ready to use set of digital camera software applications for Unix-like systems. Install photo2 in Windows 7. http://science.wonderhowto.com/how-to/make-your-own-homemade-glow-sticks-0146580/" target=_blank>Home Made Glow Sticks. Yes I got side tracked with another project. Glow In The Dark Cocktails. Supposedly Tonic Water (quinine) glows in black light. Same with turmeric (yellow green light).
LibXtract: a simple, portable, lightweight library of audio feature extraction functions. The purpose of the library is to provide a relatively exhaustive set of feature extraction primitives that are designed to be 'cascaded' to create a extraction hierarchies.
libCinder: provides a powerful, intuitive toolbox for programming graphics, audio, video, networking, image processing and computational geometry.
3D Underworld: has an open source structured light scanner.
Real3D Scanner: make your own 3D scanner with a single camera and a video projector
Structured Light 3D Scanning: tutorial
SDL Simple DirectMedia Layer: C based portable library
SFML Simple and Fast MultiMedia Library: c++ based portable library
IAANIX: a graphical open-source sequencer for digital art
Blendnik - Real Time Performance System Using Blender and Pure Data.
OSC - Open Sound Control: the successor to MIDI
Youtube video: Create Audio Visualization
Neuromixer: VJ Software
VJsTVVJ Jumping off point.
VJ Forums
VJ Software Links
Hubble: Video source media
Prelinger Archives: Video source media
Livid: MIDI performance decks and software
Synthtopia: portal devoted to electronic music.
Geometric Tools: source code for real-time computer graphics and physics, mathematics, geometry, numerical analysis, and image analysis.
Processing Sketches
Voodoo Camera Tracker: tool for the integration of virtual and real scenes
NVIDIA Video Codecs: NVENC API supports H.264 dedicated hardware video encoding introduced in NVIDIA’s Kepler-class of GPUs
Muonics Art Design: some generative graffiti, interactive experiments, audio space, ...
Poisson Surface Reconstruction: Very relevant to how meshlab does it's stuff.