GoPro + Cinelerra: High framerate slow motion and difference key

I was recently given the GoPro Hero3 Black edition by my very kind family for Fathers’ Day. Mostly I plan to use this for its high framerate capabilities to develop my skills in aggressive inline skating (it’s very hard to know how to improve your technique when you can’t see what you’re doing).

As a Linux (Ubuntu) user my software of choice for video editing is Cinelerra (I use the cinelerra-cv package maintained by Nicola Ferralis).

Cinelerra is brilliant … but infuriating. So for anyone else out there trying to get the most of their GoPro camera while working in a Linux environment, I’ll post my experiences to try to help.

My first tinkering with Cinelerra and the GoPro revealed one nice thing: By choosing “Microsoft AVI” as the file format and “MPEG-4 Video” as my compression codec when rendering from Cinelerra I seemed to get fairly fast performance processing GoPro source files for simple things like cropping, editing and dropping frame rate. That changes dramatically when video effects and transitions are applied, though…

My first hurdle was in trying to edit some footage from the skate park in which I wanted to observe my jumping technique in slow motion. I had recorded four passes in front of the stationary camera and wanted most of the action to run full speed with just the jumps slowed to 1/4 speed.

I recorded footage at 1280x720x100FPS with a wide field of view. My plan was to output 1024x576x25FPS footage, cropping off the extra pixels to zoom in a little. For the time being I’m not worrying about audio at all.

Here’s the output footage I have produced (side note: the file I uploaded to youtube was 100FPS in the end; more on that shortly):

Challenge 1: Project framerate

My first instinct was to create a Cinelerra project with 1024×576 dimensions, 16:9 aspect ratio and a frame rate of 25FPS. I expected to be able to drop a slow-mo effect onto my timeline in the desired place to make use of all the frames in my 100FPS source to get smooth slow-mo while the rest of my footage would have frames dropped by Cinelerra to output full-speed 25FPS footage. That is how things should work.

Achieving what I pictured would have this work-flow:

  1. Create a new project in Cinelerra. (Settings->Format->Frame rate: 25; Width: 1024; Height: 576; W & H rations 1.0; Color model: RGBA-Float [for difference key; YUV-A reported to crash]; Aspect ratio: 16:9; Interlace mode: Not interlaced.)
  2. Open GoPro source files using “create new resources only” option
  3. Open GoPro source files in Cinelerra viewer (Window -> Viewer) by right-clicking Media in the Cinelerra Resources window and choosing “View”
  4. Drag slider to find start and finish points from your video (ignore error windows that pop up); click “In point ([)” and “Out point (])” buttons at start and finish points
  5. Click “to clip” and give the segment a name; repeat these steps for as many clips from the original footage as will be used in the final result
  6. Insert in-point at the start of your editing timeline; drag and drop your first clip (from the resources window) onto the timeline
  7. Fast-forward to the end of the editing timeline, move the in-point to this spot and repeat above step with your next clip
  8. Select the first section you want to make slow-motion, click the “Toggle label at current position” button – this will mark these start and end points.
  9. Note the start-time duration and end-time of the selection; change the start time to the end time, click in the duration field (the middle of the three marked in the below screenshot), type the duration your selection had andpress enter:Cinelerra selection time dialogues
  10. click the in-point and out-point buttons, marking this region
  11. Edit->Paste silence repeatedly – for half speed slow-mo paste silence once; for quarter speed paste silence three times
  12. Click on the label you added at the start of your slow-mo footage and move the in-point to this spot; leave the out-point where it is
  13. Drag and drop the “ReframeRT” video effect into the region inside the in- and out-points
  14. Make sure the “key” symbol (“generate keyframes while tweaking”) is not highlighted; Right click the “ReframeRT” bar and select “Show”. Enter your scaling factor (0.25 for quarter speed) and choose the “stretch” option

These steps should be sufficient to achieve smooth slow-motion. You have a 100FPS source (which Cinelerra does recognise as 100FPS) and a 25FPS project. The Cinelerra documentation says this of the “Stretch” option in the ReframeRT effect:

Stretch mode multiplies the current frame number of its output by the scale factor to arrive at the frame to read from its input. If its current output frame is #55 and the scale factor is 2, frame #110 is read from its input.

So… With a scale factor of 0.25 frame 2 of the output should read frame 0.5 of an input with the same frame rate. But it seems logical to me that if the input frame rate is higher, Cinelerra should be smart enough to take that into account and multiply by output framerate divided by input frame rate. It doesn’t. Checking the “interpolate” option doesn’t change anything, either. So the abve workflow achieves a jerky slow-motion effect that doesn’t include three out of every four frames in your original video, and is thus no better than having filmed with a 25FPS camera in the first place.

Cinelerra documentation talks about changing the framerate of your resources as an alternative strategy to achieve slow-mo. This would involve right-clicking the media in the resources window and choosing “Info”, then dropping the frame rate to the project frame rate. In theory you could then work in reverse, speeding up the sections of your video you want to be normal speed using ReframeRT with scale factor of 4 and leaving the parts you want slow-mo unaltered. However, in practice if I changed the frame rate of my source footage Cinelerra would only acknowledge the existence of the first quarter of the file. No good to me.

So… my solution to this issue was to change the project framerate to 100FPS (a quick way is to right-click the media in the resource window and choose the “match frame rate” option. This doesn’t alter any of your edits so is nice and safe to switch back and forth as you work on your project (in my experience so far … Cinelerra can have a habit of surprising you, though).

Working with the project in 100FPS, the rest of the workflow described above stays the same. The downside is you waste processing time rendering a 100FPS video at the end that you will then have to transcode with something like ffmpeg to drop the extra frames if you need to get down to 25FPS footage.

Challenge 2: Layering video with difference key

My particular footage involved 4 successive passes in front of my camera attempting to jump as high as I could. I really wanted to be able to compare my jumps and technique side-by-side. So I decided to try out the difference key effect.

The workflow I used was this:

  1. Have one video track (at the bottom) for the first jump and the final jump; add two videos tracks for each of the next two jumps (four additional video tracks)
  2. Leave a blank track above the track for the first jump; in the track above that, stagger the second jump clip so it starts a bit later than the first
  3. Copy a first frame from the clip of the second jump containing only the background; paste this frame on the blank track under the footage of the second jump.
  4. Drag the “freeze frame” video effect onto the track with the sigle background frame; extend the length of the effect so it is longer than the clip of the second jump; mute this track (“Don’t send to output” to the left of the track)
  5. Drag the “difference key” video effect onto both the background frame track and the track of the second jump.
  6. Right-click the difference key effect on the background frame track and choose “change”; click the “Video X: difference key” option under “shared effects”. This links the effect to the difference key effect on the footage of the second jump
  7. Right click the difference key effect on the foreground action track (the second jump) and select “show”; drag the “threshold” and “slope” options until you can clearly see the foreground action from both the first and second jump clips – just play to find what works
  8. With the recording patch turned on for only the foreground action track of the second jump, in the compositor window click the mask tool and click and drag points to mask out areas of the footage that will have no action – particularly if there things like clouds moving, as in my footage:Screenshot from 2013-09-05 21:15:37
  9. Repeat for other layered tracks

Challenge 3: Getting back to 25FPS

Now, if you’re going to play this on a reasonably-powered PC, just leave your rendered video 100FPS. But I have media players (such as my networked tv) that can’t handle 100FPS. So I used ffmpeg (note: for Ubuntu users ffmpeg has been ditched from the official packages and using the ffmpeg command actually calls a forked software package. Google to find out how to install the “real thing” because the below command won’t work otherwise):
ffmpeg -threads 0 -i CINELERRA_RENDER.avi -vcodec libx264 -crf 15.0 -acodec libmp3lame -f avi -r 25 MY_25FPS_OUTPUT.avi

This must be quality overkill because despite the quartered frame rate the files I get are as big or bigger than the 100FPS source. But at least the quality isn’t terrible so … for now that’s my method.

I hope this might be some help to some others struggling to get Cinelerra to play nice with high FPS source material.

About shortfriction

Clinical Psychologist
This entry was posted in Digital video, Linux and tagged , , , , . Bookmark the permalink.

9 Responses to GoPro + Cinelerra: High framerate slow motion and difference key

  1. Pingback: Cinelerra workflow – making Cinelerra work despite the bugs | Proof of Geek

  2. rennrad says:

    nice howto! i have a question for you, and i believe that will be easy for you. i have two files, one with 25 fps and another with 30 fps. i tried both strategies: duplicating frames for the 25 fps file and framedropping for the 30 fps file (used ffmpeg for that), but the rendered clip always displays the 25 fps video (main camera) in slo-mo. ideas on how to overcome that? thanks!

    • I haven’t dealt with this scenario myself. I would probably first import both clips direct into cinelerra without processing with ffmpeg, and experiment with three different project settings in Cinelerra: 25 FPS, 30FPS and 150FPS (150 being the lowest common multiple of 25 and 30). While I haven’t combined 25 and 30FPS source in any timelines I have had success with projects combining 60 and 120FPS framerates. I usually use a framerate that is the lowest common multiple of whatever source framerates I am using. I have encountered a bug in cinelerra that resulted in ultra-slow-motion renders when doing a mix of 48 and 120FPS source footage. This post: https://proofofgeek.wordpress.com/2013/12/23/cinelerra-gopro-mixing-1080p-48fps-and-720p-120fps/ describes the solution to that problem.

      • rennrad says:

        funny that despite the 25fps feed seemed like out of sync the clip i rendered looked fine, without tweaking. i guess i won’t have problems as i imagined… anyway, i will check again your tips, in order to mix regular shooting with time lapses – and producing a sound track for that.

  3. Cin-user says:

    Hi!
    Please make a tutorial showing how to use:
    http://cinelerra.org/docs/cinelerra_cv_manual_en.html#SEC150
    14.1 Realtime effect types
    Shared effects and shared tracks … Who is who? How are they doing? From whom who take settings? Who with whom mixes?
    Very complicated thema…

    • Thanks for your question. I hope I can help. Let’s start with a shared effect. Say you have two video tracks, video1 and video2. You put the ReframeRT effect on video1. You right-click the effect’s bar at 0 seconds (with “Generate keyframes while tweaking” enabled) and set “scale by amount” to 1. The track will play at normal speed. Without closing the ReframeRT settings box, you move your selection to 10 seconds and change “scale by amount” to 2. video1 will now play at single speed from 0 to 10 seconds and double speed from 10 seconds on. There will be a yellow key symbol on the effect timeline at each of those points.

      Now you right-click on video2 and choose “attach effect”. A window pops up with “video1: ReframeRT” listed under shared effects and “video1” listed under shared tracks. You click on “video1: ReframeRT” and then click the green tick. Now you have an effect on video2 called “video1: ReframeRT”. It has no settings you can edit. Without doing anything more, video2 will now play at single speed from 0 to 10 seconds and double speed from 10 seconds on, same as video1. If you now add a third keyframe on the ReframeRT effect attached to video1 at 20 seconds (scale by amount 0.5), both video1 and video2 will now change to half speed from 20 seconds on. You can add other effects on the timeline of video1 – like a scale effect – and they won’t effect video2. Only the ReframeRT effect is copied.

      Now, I’ve never used a shared track. But I believe if you shared the TRACK instead of the effect then EVERYTHING you do on video1 – fades, pans, effects, etc. – will be applied identically on video2. You could then add OTHER effects to video2 as well.

      What I’m not sure about is what happens to the actual footage on the two tracks. The documentation seems to say they get mixed in some way unless you mute something. I would try playing with it to see what happens.

  4. Cin-user says:

    Thank you very much, good man!
    ———————————————–
    Now much has become clear. Not all, of course, but I have to work on it.
    I hope that you will make your continued research and write an article.
    That would be wonderful. Please write about shared track.
    Please, take as a basis of the article, section of the manual and make an article with concrete examples. 😦 All over the internet there are no how-to on the subject.

    Another interesting topic related to this issue
    http://cinelerra.org/docs/cinelerra_cv_manual_en.html#SEC136

    ” .. The order of the compositing pipeline affects what can be done with masks. Mainly, masks are performed on the temporary after effects and before the projector. This means multiple tracks can be bounced to a masked track and projected with the same mask ..”

    And this http://cinelerra.org/docs/cinelerra_cv_manual_en.html#SEC108
    ” ..If a track is part of a **shared track effect **, the output of the track with the
    ** shared track effect ** is overlaid on the final output even though it is routed back to another track (the shared track). Mute track is used to keep the track with the ** shared track effect ** from overlapping the output of the source track (the shared track) where the shared track effect is not present. See section Realtime effect types. ..”

    P.S.
    Excuse my bad English.

  5. Cincv says:

    I’ve already figured largely in this matter.
    But it is not clear what they meant, enclosed in parentheses:
    * even though it is routed back to another track (the shared track). *
    Who where redirected? How does it look?
    ====================
    Why are they in parentheses write “shared track”?
    * overlapping the output of the source track (the shared track) *

  6. Cincv says:

    What about my questions ? πŸ™‚
    Could you explain, please ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s