Photoshop software allows the user to build artwork in layers. Layer masking allows you to hide or use as much of any one layer as you like. Some people call a mask a matte, a frisket or a hi-con. It is a black and white image that cuts out a layer, allowing it to show though wherever the pixels are white, and holds out or hides a layer wherever the mask is black.
There are two basic types of masks–layer masks and vector masks. A layer mask is pixel-based. You create a layer mask either by drawing white on a black background or by building a high contrast image (a key) based on the luminance or color of the layer.
A vector mask is a geometrical shape rather than an fixed image. A vector mask is adjustable, as it is not tied to individual pixels. Create the shape of your mask using the path or shape tool. You can add points on to a path, alter the curve of its shape or size it up or down.
Make the layer on which you want to add a vector mask active by clicking on it in the “Layers Control” panel. Create a geometric shape or custom path, then transform it into a vector mask; choose “Layer” from the menu bar, then “Vector Mask” and “Current Path” from the pull-down and sub menus. Your mask icon appear next to your image icon on the "Layers Control" panel.
You can change the anchor points, shape or smoothness of a vector based shape mask in Photoshop CS3. In Photoshop CS4, you can also soften the edges of the vector mask.
Change a vector mask to a layer mask at any time by selecting “Layer,” “Rasterize” and “Vector Mask.” You can invert this rasterized (pixel-based) mask or soften it using blur filters.
Key-based Layer Masks
Create a quick layer mask using the luminance or color values of your layer’s image. Choose “Select” and “Color Range” from the menu bar and pull-down menu. Use the eye dropper tool and click anywhere on your image to create a “Key Selection.”
Click on the lightest part of your picture to “Key” (isolate) the lightest areas. To transform this selection into a layer mask, choose “Select” and “Save Selection.” Save the selection as your layer mask.
Another way to create a key-based mask is to use the “Magic Tool.” Adjust the “Tolerance” level and click on whatever area you want to define as a mask. Soften your selection by choosing “Select” and “Feather.” Save your selection as described above.
Drawn Layer Masks
Hand draw a layer mask by choosing “Layer” and “Add Layer Mask” from the menu bar and pull-down menu. Then choose “Hide All.” Your layer image is currently hidden. Reveal the image and create your layer mask at the same time by painting on your mask, now black, white. Paint with gray to control the density with which you reveal your layer.
Use masks to separate a foreground image from a background image. Use "Color Range" for images of people photographed against a blue or green backdrop. Select the blue or green color using the "Color Range" eyedropper and create a layer mask that cuts out the subject.
Paint with airbrush tools to layer in subtle atmospheric effects in landscape images or to brush glow effects onto skin.
Geometric masks help with graphic composites, where lettering or logos need to be layered in different colors on top of one another.