資訊服務Business News

Perceived as a step up from text-to-image models, such as Open AI, Midjourney, Stable Diffusion, and DALL-E, video-generation systems represent the next stage in the generative AI technology race that includes tech giants like Microsoft and Google as well as much smaller start-ups. Last month, Runway launched the mobile app version of Gen-1, which enables people to generate an AI video in minutes. While this new technology could speed up the work of moviemakers and other digital artists, it is also becoming an easy way to create hard-to-detect online misinformation. After a brief introduction of Runway – the new player in the AI technology race – and its app, this article will analyze the strengths and weaknesses of text-to-video generators as well as how this system will likely evolve.

The ability to edit and manipulate films and videos is nothing new. In recent years, researchers and digital artists have been using various AI technologies and software programs to create and edit videos that are often called deepfake videos. But systems like the ones Runway has created could replace editing skills with the press of a button. While Google and Meta decided to not share their video-generation systems with the public to prevent the risk of being used as a new way to spread disinformation, Runway’s CEO, Cristóbal Valenzuela, said he believed the technology was too important to keep in a research lab.

Runway AI, a New York-based start-up that helped develop the AI image generator Stable Diffusion, is one of several companies working on systems that can produce short videos from any short description in minutes. Over the last four years, this start-up of now about 40 employees has raised funding from top-tier investors including Felicis, Coatue, Amplify, Lux, and Compound, and has developed a suite of more than 30 online AI-powered tools. Gen-1 is Runway’s first mobile app that gives users access to video-to-video generative AI. With this app, users are able to record a video from their phones and generate an AI video in one or two minutes. They can also transform any existing video by using text prompts, images, or style presets. The free version of the app offers 525 credits (each second of a video uses 14 credits) and only 5 seconds long videos can be uploaded. Two premium plans are also available for additional credits and tools. Gen-2, an update to Gen-1 which should be released in the coming weeks, will let people generate videos based on the prompts they give the system.

How does a text-to-video generator work? Like other generative AI technologies, Runway’s system learns by analyzing digital data – in this case, photos, videos, and captions describing what those images contain. To start, you simply type a description of a scene that has some action, but not too much, like “a tranquil river in the forest”. Hit enter, and the system generates a video in minutes. Although this new tool could eventually ease the work of professional artists, it also comes with limitations. The videos are only four seconds long, blurry, and often look weird. However, by training this technology on increasingly large amounts of data, researchers are confident they can rapidly improve and expand its functionalities.

To conclude, generative AI technologies are booming. From AI text generators to image and video generators, this kind of technology is rapidly evolving to the point where Valenzuela, Runway’s CEO, has compared the current era of generative AI to the “optical toys” phase of the 19th century. The rapid advancement of AI-generated photos could suggest where the new video technology is going. Soon, experts believe people will be able to generate professional-looking mini-movies, complete with music and dialogue, and make creating a video as easy as writing a sentence. Given the remarkable pace of innovation, how can we predict what is coming next?

Source: https://www.nytimes.com/2023/04/04/technology/runway-ai-videos.html