How to properly play 3D video content

I have recently bought Panasonic plasma TV with active 3D technology and I was always fun of the highest possible movie quality. Understanding to all of it is not easy and because I could not find any proper and accurate informations on one page I have decided to summarize it here.

My first 3D video played on the TV was Avatar in Half SBS because I do not own bluray player or external MCE video player and the plasma cannot play iso bluray. After a few months I have finally found ideal piece of hardware for my TV which is new Intel NUC small PC with all in one in a small package. My first impression watching Avatar in full 3D resolution was HELL YES :) (which is 1920x1080p for left and 1920x1080 for right eye compared to 910x1080 for left and 910x1080 for right eye) - so you get twice resolution and of course more details due to original H.264 stream).

The most quality 3D content as of today is on 3D bluray discs which contains video for left and right eye. Video for the left eye is main stream which is also used for watching in 2D in 1080p resolution. Content for the right eye is encoded as a difference to the left main stream. Both stream are decoded by bluray player (either HW or SW) and if the TV is connected via 1.4a HDMI  the resolution of the HDMI is switched to 1980x2160 resolution with 24 frames/sec (this at least should do hardware bluray players). One 1980x2160 frame is left and right eye stacked horizontally. This is the best method to send 3D content over HDMI but there are several problems. First - there are only a few software video players which can play this frame packing method which is PowerDVD, Total media theatre, Stereoscopic player and all of them are not free players. The problem is that the frame packing is complicated and not easy to program which is the reason why it is not yet in main free video players like VLC, MPC-HC, XBMC etc. The second problem is that non of those players are able to play your own subtitles for 3D bluray content. This is really frustrating but there is relatively simple solution which is described later in this blog.

All of above is reason why there is another technique to code 3D which is half resolution side by side or full resolution side by side (and other variants). Frames for left and right are simply joined into 1920x1080 resolution and such video is detected and playable by any nowadays players and TVs. Drawback is that the 3D resolution (half SBS) is half compared to 3D bluray. On the other hand the size of such file is 5x smaller and 3D files are playable by smart TVs without any external players. Full SBS is 3840x1080 resolution video with left and right frame stacked but compared to bluray the final size of the video is about 50% larger and even you encode it with the highest H.264 codec the quality still does not match to bluray (at least I was not able to achieve it with the maximum H.264 setting). Playing full SBS is again not easy task because you cannot play it over standard 1920x1080p  HDMI resolution. The player has to support HDMI 3D frame stacking which can do only Stereoscopic player or PowerDVD (and others) but no free player again. At least PowerDVD and Stereoscopic players can use your own subtitles.

How to insert your own subtitles to 3D bluerays - Mount the blueray ISO, download tsMuxer (recent version) and open the STREAM/SSIF/(thelargest).ssif file. You should see two video streams in tsmuxer - MVC and H.264, audio streams and subtitle streams (PGS codec). Remove unwanted audio streams or subtitles and add your own .srt subtitle (just click add file in input). Select Blu-Ray ISO output (ISO variant is important) and simply press start muxing. The produced iso file is standard blueray iso file which you can mount as a cdrom and play in PowerDVD or any bluray player which supports HDMI frame stacking.

Color segmentation

For one of my project I needed algorithm which tells customer how many colors will be needed to print their picture on t-shirt for example. The main problem is that you cannot use CMYK colors in my case (technology limit) but separate picture into several PANTONE colors.

This is orginal picture:

I've tried reduce picture to 10 colors with imagemagick quantize function (php)

quantizeImage( 10, Imagick::COLORSPACE_RGB, 0, false, false );

and counts percentill for each 10 colors. This method gives not satisfied results for me because some pictures have dominant color (background) and it is hard to choose right threshold which colors can be droped. Look at this picture:

and try to find an appropriate threshold. If we drop colors below 1% for example we will lost two blue colors which seems to be important. If we drop colors below 0.5% we will have several not important gray colors. So color quantization is not a good way to solve my problem.

I've been searching for another approaches and found term "color segmentation" or "image partitioning". Some segmentation code is in imagemagick, but it is terrible slow and it gives totally different colors. Mean shift algorythm looks promising but I did not find usable implementation (except of java imageJ program). Finally I found mean shift algorythm in C++ by Dorin Comaniciu and Peter Meer -

http://www.caip.rutgers.edu/~comanici/segm_images.html :

A general technique for the recovery of significant image features is presented. The technique is based on the mean shift algorithm, a simple nonparametric procedure for estimating density gradients. Drawbacks of the current methods (including robust clustering) are avoided.

Their code is very old and not compilable on modern Linux distributions so I've modified it and fix some bad memory referencing and fork that project onto http://code.google.com/p/colorsegmenter/. There is source code and binary.

This program will segment our test picture into several colors (undersegmentation):

which is very nice because you does not have to mess with thresholds etc.

This program has two another modes

oversegmentation:

quantization:

If you know another implementations or techiques, please share it in discussion or send me email festr2@gmail.com.