Synchronising Sound and Animation in Anime Studio .... how??

[Mohlar]

I've just run a search of the forum, looking for tips about adding and synchronising sound tracks (music and FX) to Anime Studio animations. Most of the detailed tips seemed to relate to Moho and I know that AS has moved on a bit since then. Perhaps the best explanation was in the thread How do I add music and sound effects to my Moho animation? .... but is there any chance of a tutorial or a clear explanation of how to create an animation alongside a soundtrack that includes music and/or sound effects? It seems silly and unrealistic to add sound effects AFTER an animation has been created - so can anyone offer a clear explanation of how and when the two elements are fused? I think a lot of people will find this useful. I'd have thought that this topic merited an appearance in the main AS tutorial list as it's so integral to the whole process of animation.

[Genete]

The real difficult issue to achieve when talking about sync sound with animation is for lip sync. For that you have Papagayo and an specific sub-forum.
I have had experience creating an animation with a soundtrack of effects (in fact there were also my voice included in the final track but it did not sync to any mouth because the animation was done in first person point of view).
The best thing is to make a script or storyboard of your animation. It would make things easier to develop.
I used the following procedure for a 40 seconds animation (presentation and credits not included) for a contest. The storyboard was in my mind and the script was done in a piece of paper:
Once I had the script done (mainly my voices in a chart with approximately starting & end time for each phrase) I record them in separated files. Later upload into an audio editor (I use Audacity) and combine all together in one single file accordingly to the script. Once I'm happy with the timing I added some effects that should correspond to the moment that I want they appear. It could overlap voices as well (I don't know if it would get Papagayo crazy or not). Once I was happy with the overall soundtrack I saved it into a single file to be included into AS. There just follow your storyboard and animate the characters or the objects according to the voices or effects.

Resume: Script / Storyboard -> Record a single soundtrack -> Animate in AS.

If you have several shots or scenes you should follow the same procedure for each and combine them into a single animation using a non linear video editor (for me cinelerra). There you can add the most background and continuous soundtrack that can be the music if any.

Hope it helps you.
Best
-G

[Mohlar]

The real difficult issue to achieve when talking about sync sound with animation is for lip sync ...... Later upload into an audio editor (I use Audacity) and combine all together in one single file accordingly to the script.

Thanks Genete.
That's one to print and study.

I wasn't thinking so much about lip-sync as much as my interest in eventually using basic music and SFX (when I've learned more about how to control and exploit AS), but your comments also cover most of those areas for me and anyone else who is keen to use that process.

I note that you refer to Audacity - it will be interesting to see if anyone else responds to my initial query and whether they refer to any other sound editors which they feel are particularly suitable for this process. I use several wavefile editors and SFX-makers, but haven't ever tried Audacity.

Thanks again for the fast response.

[Genete]

I use several wavefile editors and SFX-makers, but haven't ever tried Audacity

It is worthwhile.

Read this:
http://www.lostmarble.com/links.shtml
It comes from the time when AS was Moho.
-G

[andyriley]

Mohlar

Greetings from West Yorkshire....

This is how I would use Anime Studio to produce an animation with voice track and then add SFX/Music.

Anime Studio only supports a single track of audio so I'd use this for lip-synching using Papagayo (the sound track being recorded using freeware Audacity as suggested by Genete - although I've got numerous other packages to record audio).

I'd then render the scenes into AVI video file(s) that can be imported into video editor (Premiere/EditStudio/Video Studio) so that the individual scenes can be merged together with the required SFX and Music. Such packages have the ability to deal with multiple tracks of video and audio (so the SFX sound files can be accurately aligned with the required video)

IMHO - the problem may be down to the limitations of Anime Studio in supporting audio coupled with the fact that it is not really designed as a final video mastering tool.

I don't know if this is the sort of thing you'd be after but it is something that could and should be developed for the AS community.

[heyvern]

At some point you need to determine when exactly a sound effect occurs.

In live action films sound effects are done AFTER the film is shot by foley artist artists who create sound effects while watching the action.

Many high end sound editing software will open video files for adding sound effects and other sound bits like music etc. I still use a very old version of sound edit on the mac which opens QT files. I can add multiple tracks of audio directly to the video.

I would suggest you animate your action and add sound effects in post. Or animate the action, note the exact time the sound effect should occur and add it in to the master sound track.

Quite frankly... I don't even WANT my animation software to have a bunch of non animation related tools. If my sound already has sound effects I animate to that. If the animated action dictates where the sound goes I would add sound later in a dedicated application with multiple audio tracks.

If you ever plan to do anything "serious" like a DVD or film production... you will HAVE to use something besides AS to handle sound anyway to mix all the different channels and create proper stereo effects.

-vern

[montblankdesign]

I am working on the mac platform. I create my animation. record a dialogue or simple sound track. fine tune the animation. This could be a continous segment or a short scene of a specific camera. If I have multiple voices of dialogue, I do one, then the next character, each time importing a new sound file. The first becomes silent but the movements stay.
When I am done, I export the file in H.264 and import the video into garage band. From there I can add as many sound effects and tweak the adio as much as I want. I can also add incidental musc. Garage band is much easier to use than anime studio. But it is mac only.

[mooncaine]

As usual, we're getting great advice from Genete and Vern. I am doing a long project now, and I'm doing it more or less Genete's way:

Make the soundtrack, including voices and sound effects, with the script and storyboard as my guide. In a sound editor, make the finished soundtrack's first draft. Try to get most of the timing decisions done now, esp. the pace of the dialogue. Don't worry about timing of sound effects so much yet, but drop them in if you've got them. You can use Audacity -- I use it for many things -- but I used Cubase LE for this project. It has better workflow: I can set my own keystrokes and get work done quickly, and it exports as both a stereo audio file and as 2 separate files, Left and Right, in one operation. I panned my 2 characters Left, and Right, muted the Sound Effects folder of tracks [Audacity doesn't have folders for tracks like that, btw], and thus my separate files each contained just one character's voice. Perfect for Papagayo.

Next, I dropped the stereo soundtrack into iMovie, and dragged in all my scanned storyboard sketches, which I already named in numerical order using a program on my Mac called Renamer4Mac. I dragged them all to the iMovie timeline at once, then changed each image's duration on the timeline, which was more tedious than it should have been, but faster than in Final Cut Pro, believe it or not [fewer clicks; less typing]. What I got was storyboard sketches timed to the scenes, in a sort of "slideshow" I call the "animatic".

Bigger productions do more with their animatic than I. All I did with it was watch it, making notes about scenes that were probably too long or too short, and notes about when sound effects might seem to come too early or late. To get an idea of when they should happen, I acted things out while the soundtrack played. I did this work at home, so my coworkers didn't have to see that!

I made changes to the soundtrack in Cubase LE, based on my notes, and exported the tracks again.

In Final Cut Pro, I dropped the new sound tracks, and also the animatic, though I discarded its own soundtrack and just used it as a general guide to help me edit the soundtrack in Final Cut Pro.

Why Final Cut Pro? Because I needed to cut the soundtrack into short segments of about 30-40 seconds. I've learned from Vern that Anime Studio Pro can't scenes much longer without some occasional problems, and I can't risk more delays. Also, FCP will let me export audio more easily and quickly -- again, it's a workflow issue for dealing with lots of files [about 45 mins of animation, total]

I cut the timeline in FCP at logical places when possible, realizing that whereever I cut, I'd have to make a new ASP file whose first frame must match the last frame of the previous cut. That means sometimes I risked a scene as long as 1500 frames @ 24fps. Most are coming out at around 500 -800 frames. Each exported audio clip is named for the span of frames it covers in the FCP timeline: "Sc_5_1-789.aif", for example. The clip that follows that is "Sc_5_790-1359.aif", etc. I use the same names for my ASP files, by copying and pasting to avoid typos.

I have a routine for quickly exporting each of the 3 audio clips per scene; I'll go into that if need be. I think I'm getting too detailed here.

During the past 2 weeks, I used the exported clips to name each new ASP file I began, and animated each scene up to the end of the sound track. That meant the soundtrack I was hearing in ASP had all the dialogue and sound effects, making animation go quickly with no time wasted puzzling over decisions. I made those timing decisions in July.

The separate exports of Left and Right channels, which only have my 2 characters' voices, are used to make rough lip-sync using ASP's built-in method. I load each char's clip into its Mouth Switch layer. If I have time, I hope to go back over the lip sync with Papagayo, but if not, at least it's roughed in and I can go over the keyframes myself later.

I'm doing about 30 secs to a minute of animation per day this way, in a limited style a la Hanna Barbera. I'd get more done if I had more experience, probably.

Hope that long story helps!

[Mohlar]

Thanks to everyone who has added tips and explanations following my post. By browsing the forum and FAQ's, I now realise that this sound issue is one that worries or affects a LOT of people. (Makes me wonder why better sound-handling options aren't being developed - or are they???)

As a 'person of mature years' for whom retirement has brought me the time to devote to my lifelong interest in 2D animation - and as a newbie to Anime Studio - I'm really grateful for all the information.

Thanks.

(Great forum, by the way. REALLY active).

[mooncaine]

I'm no expert, but I read a lot, and I remember from my reading that soundtracks are usually completed first, and then the animators work to the soundtrack. It might not be completely final [I suppose the word "final" means different things to different productions], but the idea generally is that the sound must be more or less finished first because it's easier to animate to the sound than it is to sync sound to whatever the animation director dreams up.

This was reinforced by a recent [within the last year?] article in Post magazine about the studio that does sound work for the Family Guy cartoon.

[Mohlar]

I remember from my reading that soundtracks are usually completed first, and then the animators work to the soundtrack.

Yes - that was the whole basis for my original query and is, I suspect, the reason why audio has cropped up so often in other queries. I get the idea of a storyboard and working to that - but when sound (especially music) is a crucial element, it's hard to see how you can successfully animate a sequence and THEN attach the soundtrack.

Sound effects - yes, they can be dropped in. But say you were animating to a piece of continuous music and you wanted actions to match specific and precise moments in the ebb and flow .... how on earth can you do that without a soundtrack to work to or, at the very least, a timeline that is measured in minutes, seconds and fractions of a second?

From my perspective (admittedly as a newbie), I feel that sound options need to become a real priority for the future development of AS.

[Genete]

Sound effects - yes, they can be dropped in. But say you were animating to a piece of continuous music and you wanted actions to match specific and precise moments in the ebb and flow .... how on earth can you do that without a soundtrack to work to or, at the very least, a timeline that is measured in minutes, seconds and fractions of a second?

For that kind of sync music to the feeling of the movie (eg. thriller movies or very dramatic ones) I think there are two ways to do it.

One: Adapt the music to the animation. It would be done if you have to create an original soundtrack to the movie and it would be created for this specific movie only. It would happen to the thriller movies for example. A thriller soundtrack is normally a garbage if you ear it alone. It only have senese if it played with the image.

Two: Adapt the animation to the soundtrack. It doen's mean to animate the whole film under the restrictions of the soundtrack. It means that you make an editing and mounting of the recorded scenes to match properly the feeling of the soundtrack. Imagine the film The Lord of The Rings (part one). When the Enya's song "May it be" plays, the director mounted a lot of already recorded scenes around this splendid song due to he want to emphasize the sound more than the image. Also (I'm not sure) he make some of the recorded scenes to be playes at slow motion. Editing and mounting. On the other hand when Frodo was hunted by the dark riders (I don't remember thier names ) and finally hit by their sword the most important thing is the visual appeal and the music is adapted to it. It is an example of the other case in the same film.
Other example of adapt the animation to the music is the Rabbit of Seville (Bugs Bunny 1949). An excelent adaptation of the animation to the music. I've seen it thousand times and still laughing a lot with it.

Maybe other animation experts can give you more professional advices. Mine comes from common sense and my taste for cinema world
-G

[slowtiger]

Besides all that hassle with software and different file formats and complex workflow, let's not forget the old tricks of animation.

1. Track reading. This was done for lipsync as well as for music. Now we leave the lipsyc part to papagayo and listen to the music. Does it have a static beat? Does it get faster or slower or has breaks? Write down the beats on a long piece of paper. Any decend sound software should have a ruler which shows units as minutes:seconds:frames, so you can easily note frame numbers which is what you want to work with.

2. Bar sheets. Early sound animation used only a few, easily to animate tempi in music, which had one beat every 8 or every 12 frames. Eventually this lead to the use of bar sheets which had a vertical line every 8 frames. On these sheets directors and animators noted the planned movements, as arcs and points and notes. It turned out to be useful even in scenes without music, and a lot of animators incorporated that 8 frame count that much that all their animations were timed to an unhearable beat, making them much more impressive.

Use more paper. Use more imagination. Perfect mickeymousing like "do a movement exactly where there's a sound" should be left to machines. We are animators.

(At http://www.animationarchive.org/2006/08 ... html#links is an example of the use of bar sheets (here the more complete variety with notes and staves) complete with storyboard and movie clip. Make sure to follow the links ot other blogs in this article.)

[Mohlar]

let's not forget the old tricks of animation.

The bar sheet link is very useful. Thanks for that.
When I'm further through the AS tuorials and feel able to put together a simple bit of animation for myself, I'll try a short experiment with, say, 15-20 seconds of some well known piece of music. I'll work out everything on paper (timings, events etc.) and see what I can make work. If it's successful, I'll post a few notes and the result so that other newbies might be able to learn from whatever mistakes I make.

Thanks again.

[heyvern]

Here's a tip for you.

If the music has a "noticeable" beat to it you could use the "bone wiggle" script to insert some key frame reference points.

For example just create a bone layer and run that script on it. It will create key frames based on the audio wave form. This would give you a quick and easy reference for the timing of the music.

If the music doesn't have enough of a beat or it produces too many keys you could edit it in Audacity or something similar to exagerate the peaks and valleys of the sound... creating a pseudo "metronome" track that could be used for the bone wiggle script.

I've done that "audio exageration" editing trick before in a different application for the same sort of thing. I can see the wav form better to time the animation. I don't have to "hear" it properly because it will be replaced with the original. I just use it for visual reference.

-vern