Today on AI Daily, we kick things off with Blockade Labs' mind-blowing 3D Skybox generation tool that turns your sketches into stunning scenes. Watch as Farb demonstrates the power of this incredible tool. Then, we dive into Drag Your GAN—a game-changer for image manipulation. Discover how you can effortlessly transform images with a few simple clicks. But that's not all! Join us as we explore Meta's latest innovation, the MTIA v1 AI inference accelerator. Find out how this custom ASIC board takes AI operations to unprecedented heights. Don't miss out on this epic episode filled with revolutionary technologies and endless creative possibilities.
Main Take-Aways:
3D Skybox Generation
Blockade Labs has developed a free 3D Skybox generation tool.
Users can sketch out their desired skybox and provide a prompt.
The tool generates an entire scene based on the sketch and prompt.
It offers a quick and easy way to generate ideas and scenes.
The generated skybox can be used as a background in games or other forms of art.
The tool is accessible to anyone and can be used immediately.
It is speculated that stable diffusion and control net algorithms are used in the process.
The tool has the potential to impact 3D game developers and artists.
It simplifies the creative process by combining multiple steps into a single tool.
Users can feed the generated images into other models to create 3D assets.
The current main use case is creating skyboxes for game backgrounds.
Drag Your GAN
"Drag Your GAN" is a new GAN that allows interactive point-based manipulation of images.
Users can drag points and adjust dials to manipulate specific features in the image.
The tool enables realistic and seamless transformations of objects or faces in images.
It offers fine-tuned control over image poses and allows for easy adjustments without extensive editing.
The tool has examples demonstrating its effectiveness in manipulating cars, microscope images, faces, and animals.
It simplifies the process of fine-tuning image outputs and provides controllability beyond the base-level image models.
The ability to create masks over specific areas of an image allows for selective manipulation while keeping the rest of the image intact.
The tool is expected to be released in June, and creators are planning to share the code repository on GitHub.
It offers a more efficient and user-friendly alternative to image editing in software like Photoshop.
The tool has the potential to significantly benefit creatives and make image manipulation easier and more accessible.
Meta MTIA v1 Inference Accelerator
Meta has announced their MTIA v1, the first generation AI inference accelerator.
It is a custom ASIC board designed specifically for AI operations, focusing on specific AI math requirements.
The accelerator is different from Google's TPU and shows promise in comparison to other hardware options like etch.
It excels in handling small shapes and batch sizes, but GPUs are still more efficient for medium or large size shapes.
The integration of the accelerator with PyTorch is a significant advantage, providing backward compatibility and facilitating inference workflows.
Energy efficiency is a key feature, with the ability to run a single accelerator using only 35 watts of power.
The development of the accelerator has been in progress since 2020, indicating extensive effort and refinement.
Meta's release of this custom hardware demonstrates the growing competition in the chip race, with companies like Apple, Google, and Microsoft also investing in custom hardware solutions.
The integration of hardware accelerators like MTIA v1 has the potential to improve inference performance and reduce energy costs for a range of applications.
The future looks promising for further advancements in custom hardware for AI.
Links to Stories Mentioned:
Follow us on Twitter:
Transcript:
Conner: Good morning and welcome to another episode of AI Daily. I'm your host Connor, joined by Farb and Ethan. Today, we got three pretty great stories for you once again. Uh, starting first with Blockade Labs, who made a free 3D Skybox generation. You sketch out how you want your skybox to look, and you give it a little bit of a prompt.
And then it generates an entire scene for you. Barb, have you been playing around with this? How is it?
Farb: Yeah, I've been playing around with it. Maybe I can share my screen here and, uh, show you a little bit. This is really, this is really cool. Uh, let's see here. Hopefully y'all see my screen here so you can see this image that got generated here.
Um, and then if I hover over here, you can see these horrible little sketches. I did, you know, I did it in five seconds. I just, you know, drew a little path type thing in a couple of circles. You can see that it, uh, Uh, and some lines. It made a little mountain out of the lines. I drew up at the top and it looks like it made like a cave over here from that one circle.
Uh, we got another little moon over there from another circle I drew, and you can see this path that it kind of drew along the middle. So, uh, I previously did a little hobbit village, uh, off of this same rendering, and, uh, I thought it'd be cool. We can, it takes a little bit to get this going, so I thought we could, uh, generate a new one and, and we'll come back to it at the end of the show to, to see what it came up with.
So, uh, Connor, you wanna wanna give us a, a new style to generate this horrible little sketching?
Conner: Yeah, let's do like a polar style with some like egg glues. Polar world with igloos.
Farb: Okay, so we'll hit generate and we'll come back to this at the end of the episode and see what it came up with.
Conner: Amazing. Yeah.
So people have speculated this is using stable diffusion and control net under the hood, which would make sense. It takes different shots of it and then run stable diff fusion over that. Um, when asked themselves, Blockade Labs basically put an image of, um, Like proprietary, proprietary, proprietary, proprietary.
And everyone's just like joking, like it's like the album meme. Yeah. Um, Ethan, what, what do you think? What do you think about this? What can you do with this?
Ethan: Yeah. I started playing with it. I think we're just, we're getting more creative tools, which are always interesting. Um, I played with it. I generated a little, I.
Like sci-fi scene landscape. Um, the more creative tools we get, the better. Um, this is just sky boxes for now, but I think you can start seeing how much this will affect 3D game developers in the future. People making just scenes you may want in a movie scenes, you may want in new forms of art. Um, so the more tools we get like this, they're getting easier and easier.
You know, it used to be, okay, I'm gonna generate this image, then I'm gonna take it over to control net, edit this part, then maybe I can put it in a 3d. Kind of exploration engine, but now it's all in one and it works in Safari too. So I thought it was a fun tool, and I'm excited to see what polar ice cap we get out of farms.
Farb: I mean, you can imagine taking the images that it generates and feeding them through some other models that actually spec out the 3D assets for you. It's cool as a. It's kind of a sketch tool. You know, you can, you can lay out something quickly and have it generate all sorts of ideas. You can then take that idea and take it more seriously.
I, if you want to, I thought it was a r really fun, fun tool to play with. Go check it out. Anyone can use it right now.
Conner: Yeah. I think the main use case right now is like the sky box you have in the background of a game. You drop that into Unity, you drop that in Unreal, and then your whole background for your game is there.
So, Um, we can use this now as we've seen with farm. Um, our next story up today is drag your GAN, a very interesting new GAN that came out. Uh, interactive point base manipulation. So you get an image and then you can drag around a few points and move some dials. And then it's like a dog's head can move from one angle of the frame to a totally different angle and it flows very well.
It looks very realistic. Uh, Ethan have you, what do you think about this?
Ethan: Yeah, I think with, with this type of tools we're, you know, if you've ever messed with Photoshop and you're trying to actually manipulate portions of images or change more fine tuned poses, you'll see how long it takes. So as we get this new.
Image pipeline based on these models. You know, you're generating something with mid journey. You're generating something with stable diffusion or maybe Dolly, and you want that fine tune edit. You wanna say, oh, I just wish the head was a little bit left. They, they had some really cool examples in their paper for cars, for microscope, images, for people's faces, for animals.
So that fine tune control is, I think, what's being unlocked now. Past these kind of base level image models and so much easier for a creative to work with so much easier to say, ah, I want this, you know, controllability a little bit different than the output of the image model. And this is such an easier way than fiddling through Photoshop for an hour.
Conner: Yeah, like let's say mid journey, you generate someone's face and then, but they're frowning and move that to a smile. So yeah. Farb what do you think about this?
Farb: I am creator of screen shares, so here we go. This is really cool. So one of the things they're kind of showing off right here in particular is that you can create a mask over an area of the image and then pull that area of the image while keeping the rest of the image still.
So if they didn't put this mask on as they pulled the dog's head over to the right, what you'd see is this whole body would switch over to the right, but instead they put a mask on it, which means they can kind of just pull the head over. This is, uh, you know, imperfect, but super cool and a lot less work than, you know, editing things in Photoshop.
You know, they're, you can see here they're increasing the length of a pizza piece of clothing. Uh, really powerful stuff.
Ethan: Absolutely. Yeah. The code is not out yet, but hopefully they'll be releasing it soon. So this is something for later, but it's be cool to use when it comes out.
Conner: No, the, the repository of the creators and the read Me of the GitHub, they said released in June.
So hopefully the next couple weeks we'll see it. Um, very exciting stuff. Absolutely. Our next story up is meta. They announce their MTIA v1. It's their first generation, AI inference accelerator. So they custom asic board that they design specifically for AI operations. So GPUs are of course far better for, um, mare multiplication.
That's very important for ai. But then this is even better and focus specifically around some of the specific AI math that's needed. Um, so it's not like the TPU that comes outta Google. It's really exciting to see this coming outta meta, and we'll see how this compares to the tpu, how this compares to etch that we talked about yesterday.
Um, But yeah. Farb, any more thoughts on this?
Farb: Yeah, I thought this was interesting. One thing that they noted is that, you know, handling small shapes and batch sizes, it, it's more efficient, but for medium or large size shapes, the, the GPUs are more efficient. They're working on trying to, to get to a, a state where they're equally efficient on that.
So, you know, it's not better at everything, but it's definitely better at some things already as far as they, they're concerned. And, uh, I'll do one more fun. Uh, Screen here, you can see that they really went all out on the, on the site for this. Yes, they're, uh, they're going full Apple style here. So they're, they're taking it really seriously and they, and they want everyone to, to see that.
And it's amazing to see. And uh, this is just the beginning for them. I'm sure they're gonna be doing a lot more. Apple will be competing here, Google, Microsoft, uh, etched like we saw. Uh, it's great to see more folks creating custom hardware.
Conner: Yeah, for sure. The chip race is definitely blown up. Ethan, what do you think?
Ethan: Yeah, the one of my favorite things when I saw this is it actually integrates fully with Pie torch. Um, you know, it may not be best for all the models you run, but having this backwards compatibility and being able to say, Hey, we're gonna take a. You know, these inference workflows and we wanna run it on some of these asics, just like you mentioned, we talked about etched yesterday.
Google has their TPUs. A lot of startups are working on some new asics and meta releasing this that integrates well with Pie Torch. You can get away from GPUs and CPUs for these fine-tune workflows is important. Um, another big point they mentioned was in the stacking of these, you can run one of 'em with just 35 watts of power.
So we're getting to. You know, energy, we've talked about energy on a previous, um, podcast, and the importance of having these asics that run less energy fine-tuned for a pie torch model on some of these inference workloads, especially when you're meta size and they're rolling out these features to billions of users.
Energy costs are important and them saving probably millions of dollars, um, a month easily on some of these inference tasks that'll come down the pipeline. So cool to see new asics, cool to see new hardware and look forward to more.
Conner: Yeah, apparently it's twice as fast and I'm assuming far less energy intensively.
Ethan mentioned another clear note is they've worked on this since 2020, so I'm assuming they sped up production on this in the past few months, past year, but they've worked on this since 2020 apparently. It's very interesting. Hardware takes time. Hardware does take time. Okay.
Farb: Except for when we did Coin Mine.
We took that from idea to launch in less than 12 months.
Ethan: Oh, that's awesome.
Farb: Pros. Uh, you gotta live above a garage for a couple of years to do it, but that's the main thing.
Conner: We should tell that to the employees that met. Go back to a garage. Go back to your garage.
Farb: Yeah. We weren't creating custom silicon, but we almost, we almost got there.
Conner: We'll be back to, we'll be back to hardware one day.
Farb: Yeah.
Conner: Um, okay, so three pretty great sorts we had today. We had the blockade Lab’s Skybox. We had drag your GAN, and then of course we had Meta's, MTIA v1. Uh, what have we been seeing? Ethan, what have you been using?
Ethan: Yeah, you actually sent me a great email this morning from some of my favorite people over at Hastra.
Um, Hastra is like a GraphQL layer on top of a Postgres database or on top of the MySQL or something. So they've been working on that project for years and I saw some of their generative AI stuff. They wanna start integrating into the product and it's just awesome to see. I love Hash as a company and product.
We've used it a lot. And bringing some, some of these, you know, easier inference workloads or easier vector workloads, um, embedded into hash will be cool and just make some of these more enterprise use cases quicker.
Conner: I, I believe the main thing there was something around like, better generating your, like tables in your sequel and just like yeah, a better base for designing your whole database.
It's very interesting. That too. Yeah. Far, what have you seen?
Farb: I've been just trying to play with the new plugins in, in ChatGPT. Uh, I found one that does cool public, uh, census data so you can basically talk to census data. I thought that was interesting. You know, this is the type of stuff that you can.
This is type of stuff that's much faster to use this plugin on ChatGPT than googling around and trying to find some table somewhere on the internet that has the information that you want. So I thought it was a cool use case and probably one of the better use cases of these models right now is accessing large data sets and asking, you know, natural language questions and getting a natural language response that is not just giving you one table, but actually synthesizing the information from multiple tables and presenting it to you in a human readable format.
Conner: Absolutely. Yeah. Yeah. The plugins were interesting because I was using like Zillow's plugin and it was good and I liked it, but it's just like, in my opinion, it's a far worse experience than like using the Zillow map.
Farb: Um, yeah, I think for some use cases it's not quite there yet. Uh, and for some use cases it's already better.
Exactly. I agree.
Conner: Uh, me personally, I've been playing around with Perplexity AI. They have a new co-pilot out. Yeah. Um, it's another jump in Perplexity. We've talked about them before. One of our favorite products. They really focus on the experience of how you use G four and how you search and how you chat with it, and it's great product.
Blinking below. You guys should check it out. Um, but that's all for today. Thank you guys. Thanks for listening to See
Ethan: Polar. Let's see the Polar image.
Farb: Oh, polar. Are we ready? Let me first take a peek, a take a little peek at it. And that's right. Make sure it's, you know, cool before we take it live on air. Oh yeah, I did a pretty fun job with it.
I thought this is a, a full on sort of, uh, Santa Claus style presentation. Let's see here. Ready? Here you go. So, uh, this is the, uh, you can see the sketch lines that I drew here. So it took that path, it took the mountain, made a little shape over here, put a person there. It really likes the paths. It does really well with the paths.
It, it all on all three versions that I created. Uh, it made a path out of them. Now I just drew these silly circles here, so it's not giving it a, a whole lot to go off of, of what it should make, but it looked like it made some sort of almost Christmas ornament Little. Orbs around it. Uh, it's very Santa's workshop.
I like it. It's very Santa's workshop. Yeah, it's got a whole vibe to it. It's almost like a Super Mario world in a, looks like a, something you might see in, in Super Mario cart.
Conner: Super cool. I like it. Well check out Blockade guys. Send us reply the comments. Will you make happy to see it? Um, okay. Well, as I was closing before, thanks for listening to AI Daily.
Creativity Unleashed: Blockade Labs 3D Skybox Generation, Drag Your GAN, & Meta MTIA v1