Topic: Creating and compositing CG bricks with real footage

I'm looking for some help here. For the documentary, I'm creating series of brick-built titles (peoples' names, composited into shots from the documentary). I thought CG would be the best approach in order to match lighting, reflections, and the like accurately, but I'm not happy with the results I'm getting. My textures and lighting don't seem to match believably despite my best efforts (haven't done this sort of thing before really) and the LDraw parts are too blocky and low-detail to hold up in shots like this. Given the tight schedule for completing this film, I don't have the time to remodel and texture each brick meticulously from scratch.

I've thought about doing it in-camera, but I'm not sure if that would work well or not because of the difficulties of matching lighting and angle, not to mention compositing it into the shot convincingly.

I'm planning on a mild tilt-shift style effect on these shots, which explains the blurriness here and helps blend the composite in with the real thing a little bit.

There will be 7 of these shots in total; I'm open to any ideas on how to get good results. Are there better pre-made 3D models of the bricks that I could use to achieve a better result? (I'm currently working in Carrara). I'd be quite open to offers from any volunteers to work on the shots in return for no money, my gratitude, and film and IMDb credit on the documentary for your contribution. They need to be finished by mid-September.

Here's my stab at it. It looks "nice" in a way, but it looks like CGI. I don't want it to look like CGI.

http://i.imgur.com/LyEDx8Y.png

http://i.imgur.com/wcmcdmf.png

Re: Creating and compositing CG bricks with real footage

I think the lighting works, just roughen it up a bit. Put a rougher bump map onto the surface, also maybe add some grain and fog effects to the CG layer.

If you have the film and the CG layers done in a suitable format for After Effects/Premiere, I could have a go at compositing it in the next week or two.

http://www.cxpulp.com/attachment.php?attachmentid=874&d=1279678499
The new KB Videos coming soon.

Re: Creating and compositing CG bricks with real footage

With the tilt-shift filter, are you trying to make the person look like he would fit with the current scale of the LEGO? Since we all know that LEGO isn't that scale, we know that that's obvious CGI. One way to remedy this is to build larger, instead of using standard 2x2 bricks (for example), double up, like LEGO does at LEGOLAND and such to make their models larger. Then we would think that they might actually be there, because they look to be the right scale to the person.

However, whichever scale you choose, I agree, the CGI bricks look unrealistic. LEGO is a really hard thing to get right. You have to make the reflections visible enough that it doesn't look matte, but you have to make it look non-reflection enough to make it look real. Scratches and bump maps are what really makes it work out. Along with Bevels.

I might be able to help you out a little bit, but I don't think I'll have time to do very much...

Last edited by rioforce (July 27, 2015 (08:41am))

YouTubeWebsite
https://bricksafe.com/files/rioforce/internet-images/RioforceBiMSig.png
"Whatever you do, do all to the glory of God." - 1 Corinthians 10:31b

Re: Creating and compositing CG bricks with real footage

Pretty sure you've got the lighting wrong on the letters. It's hard to tell with the shot as it is but there's a highlight on Maxime's right leg that suggests the light should be coming in right to left, whereas the letters look lit left to right. That's one of the reasons it looks less convincing.

Lack of shadows/reflection under the letters is another. You can fake that in 3D software by making a dummy bit of wall, doing separate shadow/reflection passes and comping them into your shot.

I'd offer to help but I don't have the time or the skills to do it justice, unfortunately.

Re: Creating and compositing CG bricks with real footage

I agree with you, 0ldScratch. The lighting is wrong and the shadows are missing. The tilt-focus also makes this shot difficult to replicate, as the focus on the lettering is wrong. I understand you'd be wanting a better focus on the name, but the way that picture was shot means you'd have to have half of the name out of focus. Matching the amount of blur of the wall would help.

Re: Creating and compositing CG bricks with real footage

Thanks for the input so far!

Kieren Barnett wrote:

I think the lighting works, just roughen it up a bit. Put a rougher bump map onto the surface, also maybe add some grain and fog effects to the CG layer.

Honesty, it's already unreasonably dirty and scratchy and bump-map-y. Way more than most real LEGO. So I don't think pumping that up even higher is going to help. The only exception being I didn't get around to adding that noisey bump texture to the sloped edges yet. And there's no fog on Maxime so there's shouldn't be on the CGI, either. I do plan to add some very subtle grain and lens aberration on the letters (as well as grain on the whole shot), haven't done that yet but it wouldn't be particularly obvious at 720p anyway.

rioforce wrote:

With the tilt-shift filter, are you trying to make the person look like he would fit with the current scale of the LEGO?

Not especially. I don't necessarily want to crank it up high enough to achieve that. I was going for a (subtle) miniaturization effect.


Since we all know that LEGO isn't that scale, we know that that's obvious CGI. One way to remedy this is to build larger, instead of using standard 2x2 bricks (for example), double up, like LEGO does at LEGOLAND and such to make their models larger. Then we would think that they might actually be there, because they look to be the right scale to the person.

I should clarify, my idea is that this would be a somewhat surreal effect that will obviously be a composite of some kind. I don't want to make huge LEGO letters that actually match in scale. I'm more concerned with them looking like they're rendered and lit realistically.

I wish I could do bevels. Carrara doesn't have a good way of adding those. I probably just need higher quality piece models to resolve that issue.

0ldScratch wrote:

Pretty sure you've got the lighting wrong on the letters. It's hard to tell with the shot as it is but there's a highlight on Maxime's right leg that suggests the light should be coming in right to left, whereas the letters look lit left to right. That's one of the reasons it looks less convincing.

Lack of shadows/reflection under the letters is another. You can fake that in 3D software by making a dummy bit of wall, doing separate shadow/reflection passes and comping them into your shot.

I agree with most of these points, hadn't noticed the light on his leg.

LMDigitalMovies wrote:

The tilt-focus also makes this shot difficult to replicate, as the focus on the lettering is wrong. I understand you'd be wanting a better focus on the name, but the way that picture was shot means you'd have to have half of the name out of focus. Matching the amount of blur of the wall would help.

Nope. The tilt-shift was added in post to both the letters and the footage. The shot was shot in deep focus. So the blur matches because it was all added simultaneously.

http://i.imgur.com/wcmcdmf.png

Re: Creating and compositing CG bricks with real footage

What file formats can Carrara import? I could build names in Blender with the various bevelled bricks that I've made over the years, then export the names for you to texture and render. Is that any help?

Re: Creating and compositing CG bricks with real footage

I can't really help, but I can nitpick: It could be just me, but the letters don't seem exactly aligned with the wall. It looks as though they start further away from the edge than they end, when I compare the first M and the E.

Re: Creating and compositing CG bricks with real footage

Nope. The tilt-shift was added in post to both the letters and the footage. The shot was shot in deep focus. So the blur matches because it was all added simultaneously.

My mistake. However, I do still think something looks off with the focus.

Re: Creating and compositing CG bricks with real footage

0ldScratch wrote:

What file formats can Carrara import? I could build names in Blender with the various bevelled bricks that I've made over the years, then export the names for you to texture and render. Is that any help?

I could try working with that, yeah! Carrara supports dozens of formats. I'd say most of the standard ones but not .blend or .max. .LWO, .3ds, and .obj would all be fine I believe.

Do you have an automated way of replacing LDR bricks? Because that's been a hangup for me. I can remodel bricks to look nicer but then I'm trying to shift them into the proper coordinates, one-by-one, which takes hours with moderately complex models like these. Carrara also handles each piece as a unique master model, so I can't, for instance, automatically add slope texture to all of even one kind of slope piece at once.

Nathan wants to revise the letter models (I made this quickly for mockup purposes and the A has floating pieces right now), once we have a revised version of it I could send you what I've got.

http://i.imgur.com/wcmcdmf.png

Re: Creating and compositing CG bricks with real footage

Sméagol wrote:

Do you have an automated way of replacing LDR bricks? ... Carrara also handles each piece as a unique master model, so I can't, for instance, automatically add slope texture to all of even one kind of slope piece at once.

As far as automatically replacing LDR bricks, I've got a plan to write a Python script that will fully automate this. Right now however, you should at least be able to select all of one type of brick at a time (say all the 2x2 slope bricks), and update them to a higher quality model. You don't seem to have too many types of bricks in there, so that shouldn't be too difficult.

I'll mess around and see if I can get a video up showing how to do it.

"[It] was the theme song for the movie 2010 first contact." ~ A YouTuber on Also Sprach Zarathustra
CGI LEGO! Updated occasionally...

Re: Creating and compositing CG bricks with real footage

Littlebrick wrote:

I'll mess around and see if I can get a video up showing how to do it.

Thank you for the offer to make a video tutorial, but I feel like I can't take the time to learn Blender right now for this single purpose, simply because I don't have enough time (too many things to get done on this doc by a somewhat challenging deadline).

If somebody could get me high quality parts versions of a series of LDraw models, all fairly basic lettering like this one, that I could import into Carrara, though, I'd be very grateful! And you'd get film credit. I'm not sure what Chris Salt had in mind, it may be a similar method of replacing parts with higher quality ones.

http://i.imgur.com/wcmcdmf.png

Re: Creating and compositing CG bricks with real footage

Blender has some handy shortcuts for moving objects accurately. All my bricks have the exact same dimensions as the ldraw bricks so I can import an ldraw model and swap in the replacement bricks quite quickly. Obviously it takes longer the more complex a model is but generally it's fairly painless.

Littlebrick's method sounds even easier than what I do right now so I'd definitely be interested in seeing a tutorial video. I'm guessing it's something to do with the object data, which I've studiously avoided looking at so far.

TL;DR if you can send me some ldraw models, I can pretty them up and export as .obj no problem.

Re: Creating and compositing CG bricks with real footage

0ldScratch wrote:

TL;DR if you can send me some ldraw models, I can pretty them up and export as .obj no problem.

Thank you. After discussing with MindGame I think we might take a stab at compositing real LEGO models into the shots. I should know pretty soon if that's going to work, if it doesn't I'll contact you!

http://i.imgur.com/wcmcdmf.png

Re: Creating and compositing CG bricks with real footage

If you're interested in that tutorial Chris, I've posted it in my CGI thread here.

"[It] was the theme song for the movie 2010 first contact." ~ A YouTuber on Also Sprach Zarathustra
CGI LEGO! Updated occasionally...

Re: Creating and compositing CG bricks with real footage

Carrara controls: Ctrl+D to duplicate objects, essentially inheriting texture and any changes to master mesh applies to all.  Enable Snap to Grid via View>Grid so your pieces can move 1 inch at a time, for example.

I've kept many of my CG brick pieces in separate .blend files (one for slopes, one for bricks, plates, etc.).  Let me know if you want them or give me a list of pieces I can put together for you.

They've all been modified based on Littlebrick's inset approach.  This makes the edges rounded (if you set smooth creases) and gives the illusion of gaps between pieces without having to bevel or scale down objects.  See pics.
https://scontent-sjc2-1.xx.fbcdn.net/hphotos-xap1/t31.0-8/11119775_903164786410605_7613926524940172983_o.png
https://scontent-sjc2-1.xx.fbcdn.net/hphotos-xfp1/t31.0-8/10362824_704762926250793_3055819552041189564_o.jpg

Best format option for Blender to Carrara is .FBX.  When you import, slide Crease Angle to 90 to set all crease to smooth.

https://i.imgur.com/4b9NnS3.pnghttps://i.imgur.com/GUIl0qk.pnghttps://i.imgur.com/ox64uld.pnghttps://i.imgur.com/v3iyhE5.png

Re: Creating and compositing CG bricks with real footage

For me:

The yellows have way too much saturation for the shot.

You need some sort of integration with the wall, ie you have no shadows actually on the wall under the letters, this will be the thing that helps sell the image the most. You need to model the wall, a simple rectangle, match the lighting, and make it so there are shadows landing on the wall. Then export out a shadow map, and ambient occlusion map, so you can comp it on better.

Also, have you tried creating a HDRI image from your shot and lighting your scene that way?

Send me over an OBJ of this model, and the video file, and I'll have a whirl at comping this in for you Philip, see if I can get better results (which i think i can)

Last edited by Rsteenoven (August 2, 2015 (06:22am))

Re: Creating and compositing CG bricks with real footage

Rsteenoven wrote:

For me:

The yellows have way too much saturation for the shot.

You need some sort of integration with the wall, ie you have no shadows actually on the wall under the letters, this will be the thing that helps sell the image the most. You need to model the wall, a simple rectangle, match the lighting, and make it so there are shadows landing on the wall. Then export out a shadow map, and ambient occlusion map, so you can comp it on better.

Also, have you tried creating a HDRI image from your shot and lighting your scene that way?

Send me over an OBJ of this model, and the video file, and I'll have a whirl at comping this in for you Philip, see if I can get better results (which i think i can)

As indicated, it's really just a rough mockup so I didn't delve into compositing too heavily. Although it is lit with an HDRI generated from the shot.

I think we're going to try to do this without CGI. If that fails, though, I'll let you know! Thanks.

http://i.imgur.com/wcmcdmf.png