우리는 사람들이 실제 상호작용이 필요한 문제를 해결하는 데 도움을 주는 모델을 훈련하는 것을 목표로, AI에게 움직이는 물리 세계를 이해하고 시뮬레이션하는 방법을 가르치고 있습니다.
텍스트-동영상 변환 모델인 Sora를 소개합니다. Sora는 시각적 품질과 사용자의 프롬프트를 준수하면서 최대 1분 길이의 동영상을 생성할 수 있습니다.
프롬프트: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
프롬프트: Several giant wooly mammoths approach treading through a snowy meadow, their long wooly fur lightly blows in the wind as they walk, snow covered trees and dramatic snow capped mountains in the distance, mid afternoon light with wispy clouds and a sun high in the distance creates a warm glow, the low camera view is stunning capturing the large furry mammal with beautiful photography, depth of field.
프롬프트: A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors.
프롬프트: Drone view of waves crashing against the rugged cliffs along Big Sur’s garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliff’s edge. The steep drop from the road down to the beach is a dramatic feat, with the cliff’s edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway.
프롬프트: Animated scene features a close-up of a short fluffy monster kneeling beside a melting red candle. The art style is 3D and realistic, with a focus on lighting and texture. The mood of the painting is one of wonder and curiosity, as the monster gazes at the flame with wide eyes and open mouth. Its pose and expression convey a sense of innocence and playfulness, as if it is exploring the world around it for the first time. The use of warm colors and dramatic lighting further enhances the cozy atmosphere of the image.
프롬프트: A gorgeously rendered papercraft world of a coral reef, rife with colorful fish and sea creatures.
프롬프트: This close-up shot of a Victoria crowned pigeon showcases its striking blue plumage and red chest. Its crest is made of delicate, lacy feathers, while its eye is a striking red color. The bird’s head is tilted slightly to the side, giving the impression of it looking regal and majestic. The background is blurred, drawing attention to the bird’s striking appearance.
프롬프트: Photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee.
프롬프트: A young man at his 20s is sitting on a piece of cloud in the sky, reading a book.
오늘부터 레드팀 요원들은 Sora를 사용할 수 있으며, 피해나 위험 요소가 있는 주요 영역을 평가하게 됩니다. 또한 모델을 크리에이티브 전문가에게 가장 유용하게 개발할 수 있는 방법에 대한 피드백을 얻기 위해 여러 비주얼 아티스트, 디자이너, 영화 제작자에게도 액세스 권한을 제공하고 있습니다.
우리는 OpenAI 외부의 사람들과 함께 작업하고 피드백을 받고, 대중에게 곧 출시될 AI 기능이 어떨 것인지 보여주기 위해 연구 진행 상황을 조기에 공유하고 있습니다.
프롬프트: Historical footage of California during the gold rush.
프롬프트: A close up view of a glass sphere that has a zen garden within it. There is a small dwarf in the sphere who is raking the zen garden and creating patterns in the sand.
프롬프트: Extreme close up of a 24 year old woman’s eye blinking, standing in Marrakech during magic hour, cinematic film shot in 70mm, depth of field, vivid colors, cinematic
프롬프트: A cartoon kangaroo disco dances.
프롬프트: A beautiful homemade video showing the people of Lagos, Nigeria in the year 2056. Shot with a mobile phone camera.
프롬프트: A petri dish with a bamboo forest growing within it that has tiny red pandas running around.
프롬프트: The camera rotates around a large stack of vintage televisions all showing different programs — 1950s sci-fi movies, horror movies, news, static, a 1970s sitcom, etc, set inside a large New York museum gallery.
프롬프트: 3D animation of a small, round, fluffy creature with big, expressive eyes explores a vibrant, enchanted forest. The creature, a whimsical blend of a rabbit and a squirrel, has soft blue fur and a bushy, striped tail. It hops along a sparkling stream, its eyes wide with wonder. The forest is alive with magical elements: flowers that glow and change colors, trees with leaves in shades of purple and silver, and small floating lights that resemble fireflies. The creature stops to interact playfully with a group of tiny, fairy-like beings dancing around a mushroom ring. The creature looks up in awe at a large, glowing tree that seems to be the heart of the forest.
Sora는 여러 캐릭터, 특정 유형의 동작, 피사체와 배경의 정확한 디테일이 포함된 복잡한 장면을 생성할 수 있습니다. 이 모델은 사용자가 프롬프트에서 요청한 내용뿐만 아니라 해당 내용이 실제 세계에서 어떻게 존재하는지도 이해합니다.
프롬프트: The camera follows behind a white vintage SUV with a black roof rack as it speeds up a steep dirt road surrounded by pine trees on a steep mountain slope, dust kicks up from it’s tires, the sunlight shines on the SUV as it speeds along the dirt road, casting a warm glow over the scene. The dirt road curves gently into the distance, with no other cars or vehicles in sight. The trees on either side of the road are redwoods, with patches of greenery scattered throughout. The car is seen from the rear following the curve with ease, making it seem as if it is on a rugged drive through the rugged terrain. The dirt road itself is surrounded by steep hills and mountains, with a clear blue sky above with wispy clouds.
프롬프트: Reflections in the window of a train traveling through the Tokyo suburbs.
프롬프트: A drone camera circles around a beautiful historic church built on a rocky outcropping along the Amalfi Coast, the view showcases historic and magnificent architectural details and tiered pathways and patios, waves are seen crashing against the rocks below as the view overlooks the horizon of the coastal waters and hilly landscapes of the Amalfi Coast Italy, several distant people are seen walking and enjoying vistas on patios of the dramatic ocean views, the warm glow of the afternoon sun creates a magical and romantic feeling to the scene, the view is stunning captured with beautiful photography.
프롬프트: A large orange octopus is seen resting on the bottom of the ocean floor, blending in with the sandy and rocky terrain. Its tentacles are spread out around its body, and its eyes are closed. The octopus is unaware of a king crab that is crawling towards it from behind a rock, its claws raised and ready to attack. The crab is brown and spiny, with long legs and antennae. The scene is captured from a wide angle, showing the vastness and depth of the ocean. The water is clear and blue, with rays of sunlight filtering through. The shot is sharp and crisp, with a high dynamic range. The octopus and the crab are in focus, while the background is slightly blurred, creating a depth of field effect.
프롬프트: A flock of paper airplanes flutters through a dense jungle, weaving around trees as if they were migrating birds.
프롬프트: A cat waking up its sleeping owner demanding breakfast. The owner tries to ignore the cat, but the cat tries new tactics and finally the owner pulls out a secret stash of treats from under the pillow to hold the cat off a little longer.
프롬프트: Borneo wildlife on the Kinabatangan River
프롬프트: A Chinese Lunar New Year celebration video with Chinese Dragon.
이 모델은 언어를 깊이 이해하여 프롬프트를 정확하게 해석하고 생생한 감정을 표현하는 매력적인 캐릭터를 생성할 수 있습니다. 또한 Sora는 하나의 생성된 영상 내에서 캐릭터와 시각적 스타일을 정확하게 유지하는 여러 개의 장면을 만들 수 있습니다.
프롬프트: Tour of an art gallery with many beautiful works of art in different styles.
프롬프트: Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous sakura petals are flying through the wind along with snowflakes.
프롬프트: A stop motion animation of a flower growing out of the windowsill of a suburban house.
프롬프트: The story of a robot’s life in a cyberpunk setting.
프롬프트: An extreme close-up of an gray-haired man with a beard in his 60s, he is deep in thought pondering the history of the universe as he sits at a cafe in Paris, his eyes focus on people offscreen as they walk as he sits mostly motionless, he is dressed in a wool coat suit coat with a button-down shirt , he wears a brown beret and glasses and has a very professorial appearance, and the end he offers a subtle closed-mouth smile as if he found the answer to the mystery of life, the lighting is very cinematic with the golden light and the Parisian streets and city in the background, depth of field, cinematic 35mm film.
프롬프트: A beautiful silhouette animation shows a wolf howling at the moon, feeling lonely, until it finds its pack.
프롬프트: New York City submerged like Atlantis. Fish, whales, sea turtles and sharks swim through the streets of New York.
프롬프트: A litter of golden retriever puppies playing in the snow. Their heads pop out of the snow, covered in.
현재 모델에는 여전히 개선해야 할 부분이 남아있습니다. 복잡한 장면에서 물리적 현상을 시뮬레이션하는 데 어려움을 겪을 수 있으며, 특정한 인과 관계를 이해하지 못할 수도 있습니다(예: 캐릭터가 쿠키를 깨물어도 쿠키 자국이 표시되지 않을 수 있음). 또한 모델은 왼쪽과 오른쪽을 구분하는 등 프롬프트에 포함된 공간적 세부 사항을 혼동하거나 특정 카메라 궤적과 같이 시간에 따라 전개되는 사건을 정확하게 묘사하는 데 어려움을 겪을 수 있습니다.
프롬프트: Step-printing scene of a person running, cinematic film shot in 35mm.
Weakness: Sora는 때때로 물리적으로 불가능한 움직임을 만들어냅니다.
프롬프트: 풀로 둘러싸인 외딴 자갈길에서 다섯 마리의 회색 늑대 새끼들이 뛰놀며 서로 쫓으면서 노는 모습. The pups run and leap, chasing each other, and nipping at each other, playing.
Weakness: Animals or people can spontaneously appear, especially in scenes containing many entities.
프롬프트: Basketball through hoop then explodes.
Weakness: An example of inaccurate physical modeling and unnatural object “morphing.”
프롬프트: Archeologists discover a generic plastic chair in the desert, excavating and dusting it with great care.
Weakness: In this example, Sora fails to model the chair as a rigid object, leading to inaccurate physical interactions.
프롬프트: A grandmother with neatly combed grey hair stands behind a colorful birthday cake with numerous candles at a wood dining room table, expression is one of pure joy and happiness, with a happy glow in her eye. She leans forward and blows out the candles with a gentle puff, the cake has pink frosting and sprinkles and the candles cease to flicker, the grandmother wears a light blue blouse adorned with floral patterns, several happy friends and family sitting at the table can be seen celebrating, out of focus. The scene is beautifully captured, cinematic, showing a 3/4 view of the grandmother and the dining room. Warm color tones and soft lighting enhance the mood..
Weakness: Simulating complex interactions between objects and multiple characters is often challenging for the model, sometimes resulting in humorous generations.
우리는 OpenAI 제품에서 Sora를 사용할 수 있도록 하기 전에 몇 가지 중요한 안전 조치를 취할 예정입니다. 잘못된 정보, 혐오 콘텐츠, 편향과 같은 분야의 도메인 전문가인 레드팀 요원들과 협력하고 있으며, 모델에 대해 적대적 테스트를 수행할 예정입니다.
또한, Sora에서 생성된 동영상이 언제 생성되었는지 구별할 수 있는 탐지 분류기 등 오해의 소지가 있는 콘텐츠를 탐지하는 데 도움을 주는 도구를 구축하고 있습니다. 이 모델을 OpenAI 제품에 배포하는 경우 향후 C2PA 메타데이터(새 창에서 열기)를 포함할 계획입니다.
배포를 준비하기 위해 새로운 기술을 개발하는 것 외에도, DALL·E 3를 사용하는 제품을 위해 구축한 기존의 안전 방법(새 창에서 열기)을 활용하고 있으며, 이는 Sora에도 적용할 수 있습니다.
예를 들어, OpenAI 제품에서 텍스트 분류기는 극단적인 폭력, 성적인 콘텐츠, 혐오 이미지, 유명인 초상, 타인의 IP를 요청하는 등 사용 정책을 위반하는 텍스트 입력 프롬프트를 확인하고 거부할 수 있습니다. 또한, 생성된 모든 동영상의 프레임을 검토하여 사용자에게 표시되기 전에 사용 정책을 준수하는지 확인하는 데 사용되는 강력한 이미지 분류기를 개발했습니다.
우리는 전 세계의 정책 입안자, 교육자, 예술가들과 협력하여 그들의 우려를 이해하고 이 새로운 기술의 긍정적인 사용 사례를 파악하기 위해 노력할 것입니다. 광범위한 연구와 테스트에도 불구하고, 사람들이 기술을 유익하게 사용하는 모든 방법과 모든 악용 방법을 예측할 수는 없습니다. 그렇기 때문에 시간이 지남에 따라 점점 더 안전한 AI 시스템을 만들고 출시하는 데 있어 실제 사용을 통한 학습이 매우 중요한 요소라고 생각합니다.
프롬프트: The camera directly faces colorful buildings in Burano Italy. An adorable dalmation looks through a window on a building on the ground floor. Many people are walking and cycling along the canal streets in front of the buildings.
프롬프트: An adorable happy otter confidently stands on a surfboard wearing a yellow lifejacket, riding along turquoise tropical waters near lush tropical islands, 3D digital render art style.
프롬프트: This close-up shot of a chameleon showcases its striking color changing capabilities. The background is blurred, drawing attention to the animal’s striking appearance.
프롬프트: A corgi vlogging itself in tropical Maui.
프롬프트: A white and orange tabby cat is seen happily darting through a dense garden, as if chasing something. Its eyes are wide and happy as it jogs forward, scanning the branches, flowers, and leaves as it walks. The path is narrow as it makes its way between all the plants. the scene is captured from a ground-level angle, following the cat closely, giving a low and intimate perspective. The image is cinematic with warm tones and a grainy texture. The scattered daylight between the leaves and plants above creates a warm contrast, accentuating the cat’s orange fur. The shot is clear and sharp, with a shallow depth of field.
프롬프트: Aerial view of Santorini during the blue hour, showcasing the stunning architecture of white Cycladic buildings with blue domes. The caldera views are breathtaking, and the lighting creates a beautiful, serene atmosphere.
프롬프트: Tiltshift of a construction site filled with workers, equipment, and heavy machinery.
프롬프트: A giant, towering cloud in the shape of a man looms over the earth. The cloud man shoots lighting bolts down to the earth.
프롬프트: A Samoyed and a Golden Retriever dog are playfully romping through a futuristic neon city at night. The neon lights emitted from the nearby buildings glistens off of their fur.
프롬프트: The Glenfinnan Viaduct is a historic railway bridge in Scotland, UK, that crosses over the west highland line between the towns of Mallaig and Fort William. It is a stunning sight as a steam train leaves the bridge, traveling over the arch-covered viaduct. The landscape is dotted with lush greenery and rocky mountains, creating a picturesque backdrop for the train journey. The sky is blue and the sun is shining, making for a beautiful day to explore this majestic spot.
Sora는 확산 모델로, 정적 노이즈처럼 보이는 영상으로 시작하여 여러 단계에 걸쳐 노이즈를 제거하면서 점진적으로 변형하는 방식으로 영상을 생성합니다.
Sora는 전체 동영상을 한 번에 생성하거나 생성된 동영상을 확장하여 더 길게 만들 수 있습니다. 모델이 한 번에 여러 프레임을 예측할 수 있도록 하여 피사체가 잠시 시야 밖으로 사라지더라도 동일하게 유지되도록 하는 까다로운 문제를 해결했습니다.
GPT 모델과 유사하게 Sora는 트랜스포머 아키텍처를 사용하여 우수한 확장 성능을 제공합니다.
영상과 이미지를 패치라고 하는 작은 데이터 조각들의 모음으로 표현하는데, 각각의 패치는 GPT에서 사용하는 토큰과 비슷합니다. 데이터를 표현하는 방식을 통합함으로써 다양한 길이, 해상도, 화면비율을 아우르는 이전보다 더 광범위한 시각 데이터로 확산 트랜스포머를 훈련할 수 있습니다.
Sora는 DALL·E과 GPT 모델에 대한 과거 연구를 기반으로 구축되었습니다. Sora는 시각적 훈련 데이터에 대해 매우 상세한 설명의 캡션을 생성하는 DALL·E 3의 캡션 재생성 기법을 사용합니다. 결과적으로 모델은 생성된 영상에서 사용자의 텍스트 지시를 더 충실하게 따를 수 있습니다.
모델은 텍스트 지침만으로 영상을 생성할 수 있을 뿐만 아니라 기존의 정지 이미지를 가져와 영상을 생성할 수 있으며, 이미지의 내용을 정확하고 작은 디테일까지 신경 써서 애니메이션으로 만들 수 있습니다. 모델은 기존의 영상을 가져와서 확장하거나 누락된 프레임을 채울 수도 있습니다. 기술 보고서에서 더 자세히 알아보세요.
Sora는 실제 세계를 이해하고 시뮬레이션할 수 있는 모델의 기반이 되며, 우리는 이러한 능력이 AGI를 달성하는 데 중요한 이정표가 될 것이라고 믿습니다.