doc banner

1. Camera Models

Table 1. Table Live Streaming Cameras
Model Streaming Supported Stitching Resolution

THETA S

Yes

External

1920x1080 @ 30fps

RICOH R Dev Kit

Yes

Internal

1920x1080 @ 30fps

THETA SC

No

Not supported

N/A

The RICOH R Dev Kit may not be released.[1]

2. Software Video Devices

On your Mac or Windows computer, you have three options for THETA webcam devices: RICOH THETA S, THETA UVC HD Blender, THETA UV FullHD Blender. You’re probably wondering what the differences are between the three webcam devices. As of Jan 2017, this is my understanding of the differences. This is based on the official RICOH specifications. If this table is in error, please add to the discussion here.

Table 2. Table Live Streaming Drivers
Driver Display Format Image Size and FPS Streaming Format USB Standard

THETA S

dual-fisheye

1280x720 @ 15fps

MotionJPEG

UVC 1.1

THETA S

dual-fisheye

1920x1080 @ 30fps with firmware 01.82

H.264

UVC 1.5

THETA UVC HD Blender

Equirectangular

1280x720 @ 15fps

MotionJPEG

UVC 1.1

THETA UVC FullHD Blender

Equirectangular

1920x1080 @ 30fps with firmware 01.82

H.264

UVC 1.5

3. THETA S Live Streaming Mode

With the THETA off, press and hold the mode button. Keep pressing mode and then press power. The camera will go into live streaming mode.

mode buttons
Figure 1. Start THETA in LiveStreaming mode

With the camera in live streaming mode, the word Live will appear in blue.

live light
Figure 2. Verify that the camera’s Live light is on

3.1. Connect Camera to Computer

Plug the camera into your computer with a micro USB cable. It will appear as a normal webcam. The camera will be called RICOH THETA S.

skype webcam
Figure 3. Confirm you can see RICOH THETA S as a webcam

The THETA is now streaming in dual-fisheye mode. If you’re using firmware 01.82 or above, the stream is 1920x1080 at 30fps using H.264. Earlier firmware will produce a 1280x720 stream at 15fps

3.2. Install Live Streaming Software

To stream the video to YouTube, install the official RICOH Live-streaming app and OBS.

Many software can be used to stream to YouTube. Refer to YouTube Live Verified Devices and Software for more information.

3.2.1. Download and Install RICOH Live-Streaming App

live streaming download
Figure 4. Download the RICOH Live-Streaming App
  1. Select Windows 32bit, Windows 64bit, or Mac

  2. Turn your THETA off

  3. Unplug THETA from your computer

  4. Install software

uvcblender install
Figure 5. Live-Streaming App File is Called UVC Blender

With the THETA turned off, the software will prompt you to reconnect the THETA to register your camera.

register
Figure 6. Register THETA with computer
RICOH updated UVCBlender to 2.0.0 on November 24, 2016. Official announcement. This was a significant upgrade that enabled full HD at 30 fps with USB. It also moved to USB Video Class (UVC) 1.5. See USB device class specifications.

After you connect your THETA, a Register button will appear.

register button
Figure 7. Register buttons appears after connection

Complete the registration.

registration complete
Figure 8. Camera and computer registration complete

Test the THETA UVC FullHD Blender driver with any software that works with a webcam. In the example below, I am using Skype.

skype test
Figure 9. Testing THETA UVC FullHD Blender with Skype
In Skype, the video does not have 360° navigation (as of Jan 2017) and it will look like a distorted rectangle. Skype is for testing only, not for use.

You can also test it with other software such as OBS if you already have it installed.

webcam options
Figure 10. Testing THETA UVC FullHD Blender with OBS
Make sure you select THETA UVC FullHD Blender and not RICOH THETA S.

4. Configure Computer To Stream to Internet

Download and install OBS.

obs icon
Figure 11. OBS Studio

Create a new Scene. Any name is fine. Click on the plus sign. Under Sources, add THETA Full HD UVC Blender (any name is fine) and add a video capture device. Right click to open the pop-up menu.

obs video capture
Figure 12. Add Video Capture Device to OBS Sources

Select THETA UVC Blender as the Device. Verify that the video stream is in equirectangular format.

obs device
Figure 13. Verify THETA UVC Blender works in OBS

If you see this message, the camera is not connected properly. Go to the troubleshooting section or read the tip below. image::img/livestreaming/error-message.png[role="thumb" title="Error message when camera not detected"]

If you see a black screen that says Status:0x800705AA, try to toggle your device between your two webcams. If you still see the error, disconnect all other webcams or disable the webcam on your laptop and then reboot your computer. The error above indicates that a connection is not established. See Troubleshooting section below

Leave the Resolution/FPS Type as Device Default.

resolution fps
Figure 14. Leave Resolution/FPS Type as Default

Under Settings → Video, set the resolution to 1920x1080 at 29.97fps (30fps will also work).

obs settings video
Figure 15. Configure Resolution to 1920x1080
In November of 2016, the maximum resolution for UVC Blender got upgraded to 1080p from 720. You need the latest driver and firmware to achieve the higher resolution.

Select Stretch to screen.

obs stretch to screen

5. Create a YouTube Live 360° Event

Log into YouTube. Click on the Upload button. Click the Get started button on live streaming.

youtube livestream
Figure 16. Click Live Streaming after you click upload

Select Events.

youtube event
Figure 17. Select Events
Make sure you select Events. You will not get a 360° stream with Stream now.

In the right side of the screen, select New live event.

youtube new live event
Figure 18. New live event

Add a title.

Select Advanced Settings

youtube advanced settings

Select This live stream is 360.

youtube livestream360
Figure 19. Select This live stream is 360

Create the event. On the next screen, grab stream name from Ingestion Settings

youtube ingestion 1

Once you click on Basic ingestion information on your encoder will open up. Set your bandwidth to the highest value your network can support. In the example below, I have it set to 720p as I have only 1.5Mbps upstream at my house.

youtube basic ingestion

Copy the stream name. You will need this for OBS. In OBS, it is called, Stream key.

youtube streamname

Open OBS, go to Settings → Stream. Paste the YouTube stream name into the box on OBS called, Stream key.

obs streamkey

On the main OBS front control panel, press Start Streaming in the right hand side of the control panel.

obs start streaming

On YouTube, go to the Live Control Room and click Preview Stream.

youtube preview

You can preview the stream if you have good bandwidth. I have limited upstream bandwidth in my office. I reduced the ingestion bandwidth, making my resolution lower. Navigation in 360 works fine. My resolution is limited by my 1.5Mbps upload bandwidth.

youtube preview test

When you’re ready, start the stream.

youtube streaming

This is an example of the THETA S streamed to YouTube with faster bandwidth.

youtube 1080
Figure 20. THETA UVC FullHD Blender and YouTube Live Event

This is a screenshot from the live stream of the RICOH R Development Kit from their demo at CES in January of 2017.

youtube 1080 ces
Figure 21. RICOH R Dev Kit Live Stream at CES

The RICOH R Dev Kit was plugged into LiveShell at the event.

6. Using HDMI

Using USB output for live streaming, you will get a maximum resolution of 1080p. This is the same resolution as saving video files to your camera. If you save still images as timelapse, you can get 5376x2688, which will be displayed as 4K on YouTube.

The THETA S has an HDMI port that can output 1920x1080 at 30fps. Until November 2016, this resolution was higher than the USB port. People were using the HDMI to get higher resolution.

In order to use this signal, you need to use something like Blackmagicdesign UltraStudio for Thunderbolt.

Once you get the video stream onto your computer, it will be in dual-fisheye. Although I have not tested it, you can may be able to use THETA UVC FullHD Blender to get this into equirectangular. With the previous driver, this was definitely not possible. If you test it and get it to work, let me know.

There are third-party products such as Streambox Cloud Encoder or MimoLive.

6.1. Streambox

streambox theta

This is the workflow.

streambox workflow
streambox sample

This is the equipment and service list used:

  • Streamed live using Streambox Cloud Encoder

  • RICOH THETA S Camera

  • BlackMagic UltraStudio Mini Recorder

  • MacBook Pro with USB Modems

  • Streambox Cloud

6.2. mimoLive

Boinx Software offers mimoLive.

They have a good video that provides an overview of their service specifically for the THETA S.

mimoLive can accept a USB or HDMI stream in dual-fisheye.

In order to use the HDMI output from the THETA, you will need a HDMI video grabber. Boinx Software recommends the Blackmagic Design UltraStudio Mini Recorder for Thunderbolt or the Magewell USB Capture HDMI adapter for USB 3.

hdmi usb
Figure 22. Getting THETA S HDMI Output Into Your Computer
dual fisheye
Figure 23. mimoLive dual-fisheye before conversion to equirectangular

mimoLive can take the THETA S dual-fisheye video stream source and apply a filter convert it to equirectangular for streaming to places like YouTube Live 360 events.

sources thetas
Figure 24. Preset configuration and filter for THETA S
placerlive
Figure 25. Filter converts dual-fisheye stream to equirectangular

mimoLive provides sliders to adjust the sphere stitching. You’ll only be able to get a good enough stitch. The edges of the spheres will not match perfectly.

adjustment
Figure 26. Use sliders to adjust sphere edges

This is an example of the 360 live stream. The quality of the stitch is good.

Even if you are using USB output, you still may want to use mimoLive instead of the free RICOH THETA UVC Blender app to take advantage of mimoLive features to add text, Twitter, and slides into the YouTube live streaming event.

text placement360
Figure 27. Place text into live stream to YouTube
twitter
Figure 28. Insert Twitter into live stream to YouTube
slides
Figure 29. Insert presentation slides into 360 live stream

You can also center your video stream.

adjust center
Figure 30. Center 360 live stream

6.3. Other

Videostream360 offers a service to use THETA at 1920x1080 with HDMI. They even sell the THETA on their site.

If you have a solution for HDMI 360° streaming and you’ve verified that it works with the THETA, please join the THETA Ecosystem and post information about it.

7. Problems with VLC 3.0.0

VLC 3.0.0 is currently not released. As of January 18, 2017, nightly builds will not play the THETA stream correctly. The stream is upside down and the color is off.

Problems with VLC 3.0.0 nightly builds

8. Tethered Streaming with Unity

Please refer to this separate article on using Unity with a tethered THETA.

The article was written prior to the release of UVC Blender and UVC FullHD Blender. It is still relevant. As of January 2017, we can’t get Unity to recognize the UVC FullHD Blender webcam.

uvc blender webcam failure
Figure 31. Unity failure to find webcam

9. Untethered WiFi Streaming

Streaming from the THETA using WiFi is primarily of interest to developers and hobbyists.

9.1. Using Unity

The THETA can live stream a 640x320 MotionJPEG at 10fps over WiFi. This is intended to preview a picture prior to taking the picture. It’s not intended for headset navigation. The community has built some solutions to stream this low-res, low fps video to mobile phones, primarily using Unity.

This is a short Vine video of a demo.

wifi unity
Figure 32. 360° video stream using WiFi

Most developers have challenges processing the MotionJPEG stream.

Fortunately, sample code of a THETA S WiFi streaming demo with Unity was developed by community member Makoto Ito. I’ve translated the README to his code as well as a related blog written by @noshipu, CEO of ViRD, Inc for his contribution.

9.1.1. About the RICOH THETA API

In order to use Wifi live streaming, you must use the _getLivePreview API. Official Reference

NOTE from Craig: This was replaced by getLivePreview in version 2.1 of the API. This blog by Noshipu-san refers to the 2.0 API, which is still supported by the THETA S. Be aware of the differences in your code.

Unlike the other APIs, _getLivePreview is different because the data is in a stream and keeps going. You will not be able to get a WWW class to wait until the request is complete (maybe).

NOTE from Craig: This is the major problem developers have when working with getLivePreview. As the data is a stream, you can’t want for the data to end before running your next command. For example, it’s different from downloading and displaying an image or video file because you know when the transfer is complete.

9.1.2. Processing Flow

Set the POST request to create a HttpWebRequest class
string url = "Enter HTTP path of THETA here";
var request = HttpWebRequest.Create (url);
HttpWebResponse response = null;
request.Method = "POST";
request.Timeout = (int) (30 * 10000f); // to ensure  no timeout
request.ContentType = "application/json; charset = utf-8";
byte [] postBytes = Encoding.Default.GetBytes ( "Put the JSON data here");
request.ContentLength = postBytes.Length;
Generate a class of BinaryReader to get the byte data (you get the bytes one by one)
// The start of transmission of the post data
Stream reqStream = request.GetRequestStream ();
reqStream.Write (postBytes, 0, postBytes.Length) ;
reqStream.Close ();
stream = request.GetResponse () .GetResponseStream ();
BinaryReader reader = new BinaryReader (new BufferedStream (stream), new System.Text.ASCIIEncoding ());
Get the start and stop bytes of 1 frame of the MotionJPEG and cut out one frame

With the byte, check the partion value of the MotionJPEG.

...(http)
0xFF 0xD8      --|
[jpeg data]      |--1 frame of MotionJPEG
0xFF 0xD9      --|
...(http)
0xFF 0xD8      --|
[jpeg data]      |--1 frame of MotionJPEG
0xFF 0xD9      --|
...(http)

Please refer this answer on StackOverflow to How to Parse MJPEG HTTP stream from IP camera?

The starting 2 bytes are 0xFF, 0xD8. The end bye is 0xD9

The code is shown below.

List<byte> imageBytes = new List<byte> ();
bool isLoadStart = false; // Binary flag taken at head of image
byte oldByte = 0; // Stores one previous byte of data
while( true ) {
    byte byteData = reader.ReadByte ();
if (!isLoadStart) {
    if (oldByte == 0xFF){
        // First binary image
       imageBytes.Add(0xFF);
    }
    if (byteData == 0xD8){
       // Second binary image
       imageBytes.Add(0xD8);
       // I took the head of the image up to the end binary
       isLoadStart = true;
    }
} else {
    // Put the image binaries into an array
    imageBytes.Add(byteData);
        // if the byte was the end byte
        // 0xFF -> 0xD9 case态end byte
        if(oldByte == 0xFF && byteData == 0xD9){
            // As this is the end byte
            // we'll generate the image from the data and can create the texture
            // imageBytes are used to reflect the texture
            // imageBytes are left empty
            // the loop returns the binary image head
            isLoadStart = false;
        }
    }
    oldByte = byteData;
}
Texture Generation Separated by Byte

This is the byte to reflect the texture.

mainTexture.LoadImage ((byte[])imageBytes.ToArray ());

Portion of Python code taken from StackOverflow answer.

import cv2
import urllib
import numpy as np
    stream=urllib.urlopen('http://localhost:8080/frame.mjpg')
    bytes=''
    while True:
        bytes+=stream.read(1024)
        a = bytes.find('\xff\xd8')
        b = bytes.find('\xff\xd9')
        if a!=-1 and b!=-1:
            jpg = bytes[a:b+2]
            bytes= bytes[b+2:]
            i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8),cv2.CV_LOAD_IMAGE_COLOR)
            cv2.imshow('i',i)
            if cv2.waitKey(1) ==27:
                exit(0)
Mjpeg over http is multipart/x-mixed-replace with boundary frame info and jpeg data is just sent in binary. So you don't really need to care about http protocol headers. All jpeg frames start with marker 0xff 0xd8 and end with 0xff 0xd9. So the code above extracts such frames from the http stream and decodes them one by one. like below.
...(http)
0xff 0xd8      --|
[jpeg data]      |--this part is extracted and decoded
0xff 0xd9      --|
...(http)
0xff 0xd8      --|
[jpeg data]      |--this part is extracted and decoded
0xff 0xd9      --|
...(http)
Testing WiFi Streaming

You can test out WiFi Streaming without having to program. Download and install Unity Personal Edition. It’s free.

Get Makoto Ito’s code for ThetaWifiStreaming.

Press Play.

game view crop
Figure 33. Unity WiFi Live Stream in Game Mode
scene 4 crop
Figure 34. Unity Scene View of WiFi Live Stream
top down sphere
Figure 35. Top down view of sphere with THETA camera positions

9.2. Using a Raspberry Pi

A Raspberry Pi can take the video live stream from the THETA using USB and transmit the stream to another device using WiFi. This is intended for software developers to use as starting point.

There is sample code available for both the transmission of the live stream and the conversion of the live stream into a navigable 360 video. Both the browser and the server applications are written in JavaScript. The server application uses node.

thetaview fisheye
Figure 36. video stream prior to conversion

The sample code uses JavaScript to convert the dual-fisheye video stream into a navigable 360° video. Transmission uses WebRTC.

thetaview 360view
Figure 37. stream conversion done in browser

10. FAQ

Q: What’s the Resolution and FPS?

A: Updated Oct 2016.

Table 3. Table THETA S Live Streaming
Type Format Camera Mode Size Frame Rate Connection

Live View

Equirectangular in MotionJPEG

Image Only

640x320

10fps

WiFi

USB Live Streaming of dual-fisheye

Dual-fisheye in MotionJPEG

live streaming

1280x720

15fps

USB Video Class 1.1 or 1.5

USB Live Streaming firmware 01.82 or above

Dual-fisheye with H.264

live streaming

1920x1080

30fps

USB Video Class 1.5

HDMI live streaming of dual-fisheye

Dual-fisheye in uncompressed YCbCr

live streaming

1920x1080, 1280x720, 720x480

30fps

HDMI

USB live streaming of equirectangular

Equirectangular in MotionJPEG

live streaming

1280x720

15fps

USB


Q: Can I stream from a drone to a headset?

A: Only with expensive equipment. This is not a good use of the THETA for recreational hobbyists. Refer to this article for more information.


Q: Does the THETA have auto-stabilization?

A: No. You’ll need to use a third-party gimbal.


Q: Is anyone using the THETA 360° stream for object recognition?

A: Yes. Most people use the raw video from 2 fisheye spheres. Most people do not convert to equirectangular video. Just extract a portion of the sphere and perform the image recognition or measurement on that section. The HDMI stream has higher resolution. Most people are using that and extracting a frame, then performing the calculation. Known applications include facial recognition, audience emotion recognition, autonomous vehicle operation. As just one example, the winner of the RICOH prize at the 2016 DeveloperWeek Hackathon used the Microsoft Emotion API on the dual-fisheye spheres.


Q: Is anyone working on panoramic sound?

A: Yes. There are many projects for 3D sound, including SOPA, an open source JavaScript library.


Q: How do I increase the sound quality?

A: Use an external microphone and add it to your mixer. Set the THETA’s input to zero using your mixer. If you’re using OBS for the stream, plug your microphones into your computer and then add a new audio source from the main dashboard to your stream. There is no way to plug a microphone directly into the THETA.

mixer
Figure 38. OBS mixer

11. Troubleshooting

11.1. Streaming to YouTube

11.1.1. Problem: Status:0x800705AA

obs error
Figure 39. Error message when device not detected
  1. Verify your firmware is 01.42 or above

  2. Make sure your camera has the blue word Live in LED lights on

  3. Toggle between webcam and UV Blender. If this still fails to resolve the problem, disable all other webcams and reboot

  4. Try a different USB cable. Plug it into the port on the back of your computer

11.1.2. Problem: Screen is black with nothing on it

Check video resolution. Set to 1280x720

11.1.3. Problem: Video on YouTube is Equirectangular with No Navigation

If the stream is in equirectangular on OBS and it can’t be navigated on YouTube, check your YouTube configuration.

11.2. Heat

The unit below overheated 16 minutes into the shoot. It is using UVC Blender and a USB cable during an indoor shoot at Stanford during a crowded VR event.

overheat example
Figure 40. Overheating during livestream

If the THETA is overheating, point a standard household fan at it. The airflow may be enough to cool the outside of the THETA and help with the internal overheating.

People have reported success by sticking $6 Raspberry Pi heatsinks onto the body of the THETA or taping or attaching a small fan used for computer CPUs to the outside of the THETA.

heatsinks
Figure 41. Raspberry Pi Heatsinks (L), small computer fan bracket ®

The enthusiast below created custom cases in plastic through a shop in Akihabara. He wanted to use metal, but the cost was too high.

case mod
Figure 42. Not recommended, but an example of community enthusiasm

12. Discuss

If you have questions, comments or additions, please post them in the THETA Unofficial Guide Forum.

If you have a product or service that you’ve verified works with the THETA, please join the THETA Developer Ecosystem. Once you’ve joined the ecosystem, you can post your product information in the Ecosystem Discussion category.

13. Return to theta360.guide


1. RICOH R Dev Kit is expected to be availabile in April 2017