Master AI video-generation prompts

Hi!

AI video-generation tools like Luma's Dream Machine, Runway Gen-3  and invideo AI are opening up exciting new possibilities.

Even complete beginners can create impressive and professional-looking clips and videos. 

As with text and image generation, a well-crafted prompt is key to getting a high-quality result. But without a background in video production, animation, or cinematography, it can be hard to know what information to include in prompts.

The prompting tips for text-to-image generation generally apply, but with the additional complexities of movement, timing, and audio, there's a lot more to consider. 

So here are some prompting tips to make the process easier.

1. Set the scene

You’re not just specifying what elements should be included in the video but also what's happening. Is there movement, interaction or a dynamic background?

What / who / where?

• Define the environment, characters, and objects

• Use specific, descriptive language where possible e.g. instead of “a person walking" use "a tall woman in a red dress striding purposefully”

Example prompt: A bustling city at sunset, a tall woman in a red dress striding purposefully, cars driving by  

Describe action and movement

• Specify how characters or objects move e.g. slowly, gracefully, fast-paced

• Choose whether the camera is static or moving  

Example prompt: The camera pans slowly from the sky to a close-up of a young boy's face as he smiles

2. Specify style and mood

Style

• What style video are you trying to create? Do you want a realistic video or are you aiming for something more abstract or cartoonish? 

• It might help to reference specific styles e.g. create a scene in the style of Wes Anderson or reminiscent of Van Gogh's Starry Night.

Example prompt: A 3D animated city scene with vivid, exaggerated colors in the style of a Pixar movie

Mood

• What sort of atmosphere or mood do you want to convey? Serious, moody, joyful, humorous?

• How might color or lighting be used to achieve this? 

Example prompt: A serene, soft-lit atmosphere with calm music

3. Include technical details 

Visuals

With more advanced tools and longer videos, you may be able to include technical details such as camera angles, pacing and transitions.

• Consider perspectives e.g. zoom in, aerial view      

• How should the pacing feel? Slow and contemplative or quick and dynamic?      

• Specify transitions between scenes e.g. fades, cuts

Example prompt: Fast-paced action scene with quick cuts and zooms

Sound 

Adding audio to AI-generated videos can enhance the overall experience. Some tools offer audio libraries and others allow you to add your own audio. Consider: 

• Background music

• Sound effects

• Voiceovers 

Example prompt: Uplifting background music with gentle bird sounds

4. Practice and iterate

The more you practice, the more you’ll find what works for you.

Experiment

Experiment with different prompts. Try keeping a prompt journal to track successful and unsuccessful attempts.

Refine

Iteration may be needed to achieve your desired outcome so be prepared to refine. Start with a basic description then, based on the output, add details and tweak the prompt to perfect your video.

Over time you might move on to advanced techniques like prompt chaining or using multiple prompts for complex scenes.

5. Keep learning

Keeping up to date with the latest tools and developments helps you to see what’s possible with current technology.

Take inspiration from others - follow other creators to see how they’re using and combining different tools.

Some creators are having success by starting with an image and animating that. Take a look at this video that Nicolas Neubert was able to create using Runway Gen 3 Alpha. 

The prompt used: static camera, fast natural movement, dynamic motion, fashion portrait, she smiles while she talks, blinking eyes 

Hopefully, this gives you a solid foundation to start creating your own AI-generated videos.

Start experimenting with your own prompts and you might be surprised at what you can achieve. Spend some time practicing and refining your skills and you'll soon be creating impressive AI-generated videos that bring your ideas to life.

OK, speak soon!

Inge
Top Apps Founder

P.S. Check out the Top Apps AI courses here.