As I Recently Predicted, Video Editing Models Will Soon Be Released. One of the First Steps is Gen-1.

As I recently predicted, video editing models will soon be released. One of the first steps is Gen-1.
On February 6th, Runway AI launched Gen-1, a new neural network designed for generating video based on text input. Think of it as Dall-E, Midjourney, or Stable Diffusion, but instead of generating images, it generates videos directly. It is a diffusion model for generation based on visual or textual descriptions. The Gen-1 model does not generate video solely from text descriptions; it requires a sample video as input, which it then modifies according to your text description or image.
The paper discusses four main features:
• Stylisation – the model modifies the video to match the style of your image
• Storyboarding – instead of placeholder objects, it generates what you need in the video
• Masking – it identifies objects in the video and modifies them according to your wishes
• Rendering – it enhances the basic 3D model with new textures, lights, and shadows
Unfortunately, the model is not yet available for testing, but you can sign up for the waiting list or view sample videos.
Sources:
https://research.runwayml.com/gen1?utm_source=RunwaySocial&utm_medium=YouTube&utm_campaign=Gen1Launch&fbclid=IwAR0j6ttNjoE0OETfWReSpk1utUtfax0mOzQgvyq8-4dtE064obrDoSBRrxM
Paper: https://arxiv.org/abs/2302.03011

#Gen1 #video #Runway #diffusion

Originally published on Facebook — link to post