Creating Plants and Trees in Maya and Mental Ray

Making the Most of Maya's Paint Effects Engine

Look around you. Unless you're deep in the heart of a windowless building (or live in the post-apocalyptic future), there's a pretty good chance that from where you're sitting you can see at least one example of foliage. Grass, trees, bushes, stray leaves in the streets, potted flowers—like it or not, plant life is present in just about every outdoor environment on the planet.

But how many CG scenes have you seen that ignore foliage altogether or treat it as an afterthought?

Due to some common misconceptions about Maya's paint effects engine (and the fact that simulating accurate, believable CG plants can be a relatively complicated pursuit), a lot of beginners either avoid it entirely or use antiquated/game-industry workarounds (like a cutout image on a 2D plane).

If you're interested in digital environment creation, and want to elevate your work to a professional level, at some point you're going to need to get a handle on some current best-practices for creating plants, trees, and ground-cover in your 3D package—in this case, Maya.

Thankfully, we're at a point where there are a lot of very good, and artist friendly foliage solutions available to us as, and believe it or not Maya's paint effects tools are right up there with the best of them.

In the rest of this article, we'll focus on some of the options that Maya users have for plants, trees, and other types of foliage, and present a short tutorial on using Maya's Paint effects engine with Mental Ray.

Software Options

Overhead view architect working at computer
Hero Images/ GettyImages

First let's take a look at some of the Maya-compatible software/plugin options for plants and foliage.

Depending on your budget, it's possible that a lot of the top plant and ecosystem generating software will be out of reach unless you work for a studio that's willing to pony up for a license. These solutions probably aren't going to be the answer for most of us “average Joe artists,” but it's good to at least know what's available, and it may even be worth your while to download the free-trials just to see what the different workflows are like.

  • Onyx Tree - Onyx is the most affordable of the major plant offerings that play nicely with the Maya/Mental Ray combination. Because you can buy Onyx assets in small modules based on species type (broadleaf, conifer, groundcover, etc.) you might be able to get away with spending as little as $195 to suit your needs. Onyx Trees look great and offer a lot of versatility through customization, making this a really good option if you're a working freelancer.
  • Xfrog - Xfrog has a module based price scheme similar to what Onyx is using, however the software still starts at approximately $399, and Xfrog trees are notoriously slow at render time. It might be worth it to downloading the trial so that you can see for yourself, but most freelancers seem to prefer Onyx.
  • SpeedTree - SpeedTree offers a lot of flexibility, and there's no denying that the trees look good, but with a base price at $900 the tool is probably out of the question for most non-working artists, with one notable exception—game development. If your main focus is in games and your software of choice is UDK, then you're in luck because SpeedTree is integrated directly into the engine. However, SpeedTree for games doesn't support quad output, so don't plan being able to bring your foliage back into a high-end package like Maya—this is UDK only.
  • Vue - Vue is decent-to-great if you know what you're doing and you're working on a lot of wide shots where the environment plays a role, but isn't necessarily front-and-center. Unfortunately, Vue trees just aren't good enough for foreground placement, but as background assets they can be pretty effective.

    Unfortunately, Vue integration with Maya and Mental Ray is questionable, the software is expensive, and Vue's render times are nightmarishly long. However, if you think this is the way you want to go, Eon does offer an unlimited personal learning edition for non-commercial use.

Non-Maya Options


I know this article was aimed at Maya users, but if you've got some crossover experience in 3DS Max, the GrowFX plugin might be worth checking out. I haven't used it myself, but it does come highly recommended.

So what if none of those options fit your budget?


Luckily, Maya has a perfectly acceptable foliage solution available to us right out of the box, assuming you know how to use it correctly. Make the jump and join us in part two, for a short tutorial on how to make Maya's paint effects plants render properly in Mental Ray:

Part 2: Making the Most of the Maya Paint Effects Engine