Welcome to AI Daily! Join us as we delve into three incredible breakthroughs that are revolutionizing the world of technology. First up, we dive into the groundbreaking world of image-to-3D conversion. Next, prepare to be captivated by the power of drones. We uncover an awe-inspiring application where drones perform real-time video analysis, tracking hundreds of objects with precision. Last but not least, get ready for a dose of CG magic! We present Wonder Studio, a revolutionary platform that can replace real people in live-action scenes with CG representations.
Key Points
1️⃣ Any Image-3D
An image-to-3D conversion tool is generating significant interest, causing delays due to high demand.
The tool allows users to input an image and obtain a usable 3D representation, saving time and effort in 3D modeling.
The tool's ability to create 3D models for applications in Unity, Unreal, and Blender is a major breakthrough, enhancing productivity and accessibility in the field. Comparison to OpenAI ShapeE suggests potential improvements in performance.
2️⃣ AI Drones
A video showcases drones performing real-time video analysis and tracking of cars, raising questions about the feasibility and technology behind it.
The video, shared by a Twitter personality, highlights the potential of drones for comprehensive tracking and analysis, although details about its real-time capabilities are limited.
The demonstration indicates a significant advancement in object recognition and image detection on consumer-grade drones, offering affordable access to real-time video and tracking capabilities that were previously limited to expensive equipment.
3️⃣ WonderStudio
Wonder Studio, a platform that can replace real people in live-action scenes with computer-generated representations, creating humorous and impressive results.
The hosts share a processed clip from The Office, featuring a CG representation of Robert California delivering a funny and unexpected dialogue.
Wonder Studio is praised for its capabilities, allowing users to achieve in hours what would have previously taken teams days or weeks, and offering powerful tools for professional video workflows, including commercial usage.
🔗 Episode Links
Follow us on Twitter:
Subscribe to our Substack:
Transcript
Farb: Hello and welcome to AI Daily. Uh, I take a quick moment here to thank all of the folks who have subscribed. We're at over 400 subscribers now, and we wanna thank every single one of you for taking the time to do that and for supporting us here on the show. We have maybe three of the best stories we've ever had on the show before, uh, today.
Powerful, powerful stuff. I know both of these guys are moved by the work that these folks have done. Let's jump into our first story. This one is pretty impressive. This is maybe the first, you know, real world example of any image to 3d. I've got one processing rate now on their discord. I, you know, they're, they're so overwhelmed with.
With people trying to use it because it's so awesome that, uh, it's gonna maybe take a little bit longer to get the results, but this is literally drop in an image and get an actual usable 3D representation. Ethan, have you, uh, seen anything quite like this before? I mean, other than maybe a paper or something?
Ethan: Uh, we saw a paper, we saw someone make a giraffe into something basic, but it looks like they're actually making their own foundational models here. Something a little bit more applicable, a little bit more general. If you've ever tried to make a 3D model, let alone a 3D model of Ironman, you know how long this takes.
So being able to input an image and get a 3D model output out that you can put into Unity, unreal, that you could put into Blender, these are huge, huge gains. So I'm super pumped to see the one that gets returned for you and also just their own
Farb: progress. You could do one today and have your 3D printer printed out by the end of the day.
Yeah. What do you think John looks?
Conner: This looks very similar to OpenAI Shape E, so we'll see how it compares. It does seem to do a little bit better, um, but, you know, great to see more of these as always, so, okay.
Farb: All right. Onto our next story. This one is, uh, also pretty powerful stuff. Here we have drones.
Now this one I'm, you know, I'm not so sure whether this is an example of something happening in real time. Basically what we have are drones doing. Video analysis and, you know, tracking every car. We'll, we'll, we'll share this in the, in the video and, and, and in the links. But you have a drone flying over an area.
It can see every single car in the space. The, the Linus, the, the, the Twitter personality that posted this mentioned that, you know, back in the day, maybe you could track a few things, but this is tracking hundreds and hundreds of things and not sure if it's happening. In real time, there wasn't a ton of information or if this is just, you know, a video analysis that you did later based on a video recording.
I don't know, Connor, what, what do you think about this sort of stuff? Yeah,
Conner: it's pretty honestly inspirational, like significant video. Um, if you click on the link inside the tweet, it links to another site called air cortex.com, which apparently has a type form where you can sign up for it. Apparently the code will be released and it is early access right now.
So if you wanna go through all those steps, you might be able to get access to it. Okay, so
Farb: it's, it's legit. Ethan, what, what did you take
Ethan: from it? Yeah, it's definitely legit. Um, it is definitely real time video and you know, if you're sitting at the Air Force using predator drones, you're probably not impressed by this cuz you've had it for a long time.
But what is impressive about this is the fact that I bet you this is running on a $200 drone that you can buy from dgi. You can buy from anyone else. That's an entirely different landscape than a 10 million drone. So being able to just throw up your hobby drone and you get a real time object recognition and image recognition.
What's cool to me is we're seeing, you know, we've talked about on the show how smaller models, consumer hardware, you get object detection, you get video detection cheaper drones. Seeing the demos of this in culmination is mind blowing. You know, you can visualize all this stuff, but at the end of the day, this code's gonna be open source.
You're gonna be able to throw your drone in the air, in whatever city you're in and get realtime video, realtime object detection, realtime tracking, probably even soon. Realtime object permanence. Off of 200 bucks and that is a completely different society we're living in. So I just love seeing the applications of these real life demos and these real life, you know, code in person.
So really cool stuff.
Farb: Be careful throwing your drone in the air. There are actually some pretty impressive. Excuse me. Drone operating systems that are open source that you can build off of. And so in a world where you can 3D print a drone, uh, take the open source drone software, start doing things like this, the, um, you know, the police state might be coming from your neighbor, uh, before it comes from the police themselves.
Uh, pretty powerful stuff. What a, what a world we're pointed towards here. Okay. Uh, we're we're flying through these stories here, ETH Connor, what did you, uh, what did you think about the drone stuff?
Conner: I liked it. I mean, if, if we can see it, actually it'd be open source and actually on GitHub somewhere, I'll be excited to see that.
So,
Farb: until then, you remain unexcited.
Conner: Until then, I'm a bit skeptical. Like I skeptical obviously. Which part are you skeptical about the video there? Real time? I haven't seen any. Proof that it would be real time. It's a lot. Yeah. Real time. There's a lot of realtime video object detection. It's, it's unlikely to be running on the drone mode.
It's probably sinking back up to an A 100, a few, somewhere on a road. You don't need an A 100 for realtime object
Farb: detection. Why don't we have a drone off? You know, you guys can each come up with your own drone technology and we'll see whose is better.
Ethan: I'm ready. I'm ready.
Farb: Yes. When we, when we find some, when we find some free time in our day, maybe.
We'll, we'll give that a try till then. Get back to work. Um, our final story, WonderStudio. I've got this one cooking as well. Uh, I'll take a quick look right now and see if it's, if it's ready for the show. If not, we'll try and, Share it a little bit later. Oh no. Oh no, it's, it's kind of ready here.
Conner: I will
Farb: not be blackmailed by some ineffectual.
Conner: Okay. This
Farb: is, uh, this is pretty hilarious. We're gonna go ahead and just show this cuz it's, it's a little bit too fun. So I took the scene of Robert California. Um, In the office and processed it through Wonder Studio, which can replace any real person in a live action with a three, you know, a, a, a CG uh, representation of that person.
This is a little wonky cuz Robert California's on the phone while he's doing this, but it's, uh, still hilarious and very impressive.
Conner: Well,
Farb: I will not be blackmailed by some ineffectual privileged feat, soft penis debutante. You wanna start a street? Fight with me. Bring it on. But you're gonna be surprised by how ugly it gets.
You don't even know my real name. I'm the lizard gang. Well, maybe that was a little bit hardcore for the show. We'll, we'll, we'll edit any, uh, inappropriate scenes out, but that is just completely wacky that this is something you can do today. So, It took about an hour for the thing to process. Uh, and it's available today.
You can, you can use it. Uh, go check it out. Wonder Studio. What? Um,
Ethan: yeah, I got to see all the way back in January, I think, when they first released, um, some of their beta clips, et cetera. So we've all been waiting for this. I think in general, this just speaks on, I think Hollywood in general, media entertainment, just there is a lot coming, um, for them.
You know, just what you generated would've taken a team at least a few days to put together, you know, make the EGI character, make sure the movements are correct. AI's gonna completely reshape a lot of this. So really cool to see.
Farb: Not long ago, it would be probably more like weeks of a, of a team's effort, you know?
Yeah. Connor, what's your read?
Conner: It's pretty amazing Wonder Studio. It's been in a closed beta since March where pretty much anyone can get access to it through application. But it is honestly one of the most amazing platforms I've seen in video or 3D at at all. Uh, it can perfectly match lighting between the shadows, between the luster of a character and a scene.
It has a lot of layers to it and you can actually get through what would've taken a team an entire year before. You can now do through under Studio in about 24 hours probably. So absolutely. All the models they have underneath there and the platform they've built, it's very easy. You just take your video, you drag in a character, done, wait, there's your video.
So
Farb: yeah, and they've got some pretty powerful tools there that real. Uh, video people would appreciate, uh, in including letting you, uh, use it commercially. So, yeah, it's, it's not just a, a little one trick pony or a quick little demo. It, it, it's something that they're imagining people could bring into their professional workflows starting now.
Yeah,
Conner: wonder Dynamics is very good work. So congrats to the guys over
Farb: there. Pretty impressive stuff. All right, well, let's move on to the, what we're seeing. Uh, I saw a cool, uh, new setting in. In Mid journey that I haven't had a chance to use. The setting is just called weird and it takes a value between zero and 3000.
And you know, apparently it just makes things weirder. And that's a pretty neat idea in the sense that, you know, one of the things that people have dogged AI for in the past and you know, still to this day is it's. Lack of creativity or if it's lack of weirdness, if you will. And so I guess the folks at Mid Journey were like, yolo, let's just make it weirder.
And, uh, they figured out a way to to do that. And it gives you one more lever to pull on, uh, to make your. Mid journey creation's, more creative,
Ethan: pretty impress, cool that LLMs have unlocked. We can use the term weird, you know, you can't define the term weird en code. It requires this type of stuff. So it's really cool.
These just the vibes that are now parameters that they never could have been before. It's always been a pretty big knack, like knock on mid journey because yes, it looked very good, looked way better than stable diffusion, but it lacked a lot of creativity that you could see in stable diffusion.
Farb: So introducing, it's going from AI's knock to AI's, nac.
Yeah, exactly. Yeah. Powerful. What are you seeing Connor?
Conner: I saw Mosaic, of course, they've done a lot of work, um, training LLMs between MPT seven B and MPT 30 B, so they have a lot of experience there. Their L LM Foundry, which is their open source model training platform, now works out of the box on amd. GPUs, so anywhere from the Nvidia A 100 s and H 100 s.
You can now use all the AMD server GPUs like their MI two 50. And it's performance that's almost right on par with NVIDIAs. I think about 80% it says. So very exciting news if you have access to AMD chips, but not Nvidia chips. And hopefully it should even out some of the competition between those two.
Farb: I mean, 80% is I think what you would, you know, use the 80 20 rule is most of what you need and probably make it practically useful.
And three months ago, People were, you know, calling for the death of amd, uh, saying the game is over and now they're at 80%. I don't imagine it's too long before there are at least some cases where people will just use the AMDs, cuz there's maybe less competition for them. They may be more readily available, they may be more affordable.
Um, hopefully that's the case for all of us.
Conner: Yeah, I dug through some of the individual benchmarks. AMD actually seems to perform better if you just have one card. So the Nvidia cards do have a bit of the linking between them better but. If you just have one and I'm sure in more future benchmarks, AMD will actually do
Farb: better.
So Nvidia ad, whichever one of you wants to sponsor the show first, we're happy to show you over the other. Uh, no, no problem here. Ethan, what are
Ethan: you seeing? Um, yeah, our good friends at Lambda Labs who you all know we love and we all know we love, uh, they release just a great guide on how to fine tune Falcon.
So we've talked about Falcon, the open source l l m. So really great guide on how to fine tune either the 7 billion or 40 billion parameter model on their H 100 s. And what's cool is, you know, H one hundreds. Are always, in theory, a lot faster than a 100 s, but they require a lot of code optimizations, not only from Nvidia, but from the developers themselves.
So Lambda released some great ways to use Laura's quantization to make sure you can fine tune these things a lot faster. So much more to come from them probably as they release more H 100 toolkits. But if you want to go, fine tune Falcon, go to Lambda.
Farb: Nice. Go to Lambda. We love the folks over at Lambda.
Well, thanks everybody for joining us, and we'll see you soon on another episode of AI Daily.
Any Image-3D | AI Drones | CG with WonderStudio