We learnt all the kindergarten masking from brush painting to color masks – now let’s go pro, masking with channels!
This technique sometimes works well for separating a person from a plain background or for masking hair (which is one of the more difficult tasks in Photoshop).
Have you ever worked with the channels of an image? No worries if not, I’ll get you into it quickly today.
Working with the standard RGB workspace in Photoshop we have three channels available: Red, Green, Blue.
Open the copy of a color image in Photoshop. On the right side where you layers are look a few lines higher and switch from the “Layers” tab to the “Channels“ tab. Instead of the “Background” layer you should now see four layers called RGB, red, green, blue. Those are your channels.
The RGB channel is a combination of all other three channels resulting in your final image.
Excursion: Remember, usually our images are calculated as a mixture of reds, greens, and blues. Having exactly the same kind of channels for our image looks like there has to be a link. Let’s explore.
The channels simply imagine as a red, a green, and a blue layer (or three sheets of colored translucent paper), stacked as a sandwich. If we block the red and the blue, just having the green come through, well, we hopefully have a green pixel in the image. Blocking green and blue gives a red pixel. Blocking everything will result in a black pixel. No blocking at all results in white.
Wow, it seems like with a mask of hiding and allowing more or less of each channel to come through we can control the final colors in the image. That’s exactly what happens in the color channels. They define the colors and brightness of your image.
So the channels are actually masks themselves already. In the standard RGB mode each of the three grey-scale channel versions will tell Photoshop how much of the colors red, green, and blue will be allowed to break through at each single pixel – defining your color image in the final mix.
Even further Excursion: In the print press standard CMYK your image would be composed out of four color channels: Cyan, Magenta, Yellow, BlacK. The image itself will look about the same, but the algorithm behind will be different.
In the recent tutorial for mastering the curves tool we had worked in Lab mode. This is a third way of calculating the image’s colors and brightness, working with a pretty abstract modulation of channels.
All theory is of no help if not knowing how to make use of it. In general we don’t even need to understand the theory behind for creating our masks with help of channels.
Create a Channel Mask
I didn’t lose you yet? You are really brave! Hold on a bit more and a new world of masking opens up for you.
If you now click on the red channel first all the others should be deactivated (the eye icons to the left of the channels disappear except for the red channel). First thing you will discover is your image changing to being a slightly strange black and white image.
Now click on the green channel, which deactivates all others and results in a different greyscale image. Same with the blue channel, again a different greyscale image shows up.
Excursion: Each greyscale image tells Photoshop how much of the reds, greens, and blues to be allowed come through at every single pixel. In the black parts of a mask the specific color of the channel is totally blocked out, in white parts it is fully applied, in the grey parts it is something in between. Since our images are a mix of reds, greens, and blues, the three colors and their three corresponding masks translate into our final image.
For masking it is important to check in which of the channels the subject we want to mask separates best from the background.
Once you have opted for one channel as the one separating your desired subject best, let’s for example say the red one, click on this channel for activating it. Now we need to duplicate the channel by right-click using the menu or by dragging and dropping the channel on to the page icon in the bottom with the mouse. The new channel we name “Mask”.
Excursion: Sometimes it helps working with more than one channel. The hair might easily be separated in the red channel, the trousers might easily be separated in the green channel, and something else might be easily separated in the blue channel.
In each channel select the interesting parts with the lasso tool “L” or with the marquee tool “m”, copy the selection with Ctrl+C , click on your new “Mask” channel and paste the selection with Ctrl+V to this “Mask” channel. With this technique you can create even more sophisticated masks in the channels.
Improving the Channel Mask
Click the “Mask” channel to activate it and have a look at the greyscale image. Most probably the mask is not perfect from start. We need our subject to be white, and the background around has to be black – or the other way around. We can easily invert in the end with the shortcut Ctrl+i.
For first improvement make sure the “Mask” channel is still activated and Press Ctrl+L, bringing up the levels controls. Move the three sliders below the histogram left and right for increasing the contrast of your mask, at best totally separating the subject you want to apply an effect to from its background. Click OK, the level window closes again.
In this step our mask does not yet have to be perfect but it should be reasonably fine already. How to apply it to a layer?
Transform Channel into Layer Mask
First we have to ensure to make a selection of the “Mask” channel before we can bring it to the layer’s tab (remember, initially we had switched from the “Layers” tab to the “Channels” tab, now we work our way back).
Check to have our self-created “Mask” channel activated by clicking on it. For making the selection now Ctrl+Click on the “Mask” channel or drag the channel with the mouse and drop it onto the dotted circle icon in the bottom of the channels tab. Either way you should see marching ants starting to parade in your mask. These altruistic little helpers show your selection.
A very important point: When you see the marching ants selection click on the RGB channel once before continuing! This brings up your image again in correct colors. (Just test one day what happens if you forget this step – you will add a strong red, blue or green color cast to your image, depending on the channel you duplicated).
Switch over again from the channels tab to the layers tab in the menu above the channels. The ants today have a long way to go, they still should be marching. Now simply click on any adjustment layer you want to add for improving the selected parts in the mask, for example a brightness layer (the sunshine icon) or any other.
If this is all geek to you check out my previous posts where all of these steps were explained in more detail already.
As long as things went well, in the layers section a brightness layer should have been added. Ok, nothing really new up to here. But look at the mask automatically added to the layer. It is exactly the selection you made in the channels.
Tipp: In case you want to tell the ants to disappear: they seem allergic against the shortcut Ctrl+d.
Refining the Mask
Usually some more work on the mask is necessary. For example when masking a person from the background with this method we might have perfectly masked the outer line but inside the body shape there might be spots left. We want to have a clean black part here and a clean white part there.
Nothing simpler than that: we have already learnt how to improve a mask by hand. Alt+Click on the mask for making it show up big. Pick the lasso tool “L”, make a selection (move the mouse while holding down the left mouse button), and fill the selection with black or white depending on how you want to have that part of the mask (fill works with mouse right-click into the selection and opting for “Fill” in the pop-up. Background and Foreground color are the ones set in the bottom of the tools panel on the left).
Or alternatively opt for the brush tool “b” and paint in black or white depending on the part of the mask you want to improve. Make sure to paint with black or white (“x” for setting black and white, “d” for switching between both) having the brush opacity set to 100 percent.
There are even more tools for more sophisticated mask refining. Double-click on your mask in the layer and the properties panel pop-up slides in again. Click on the button “Mask Edge” and work your way through the sliders. They are pretty self-explaining. The view button in this menu by the way might be helpful to play around with, for getting a better idea what changes with moving the sliders.
Sometimes it is helpful to hide our tracks in the end with the standard properties pop-up of the mask by moving the feather slider to more to the right. Just check whether it works with your specific task.
Applying the Layer Effect
I should not need to write this anymore, and you should now shake your head about me wasting our time on such trivial things. But after having finished the mask for our layer we still have to apply the effect, in our case increasing brightness of the masked part of the image.
Well, double-click on the icon in the layer, in our example it was a brightness layer which has a sunshine icon next to the mask. Therefore double-click on that sunshine icon, the properties pop-up slides in, and here we increase the brightness, which should be now applied on the white parts in the mask only.
Ooops, we masked the wrong way, having the effect applied exactly on all parts we wanted to exclude it from? No problem, you already know: Click on the mask once, Ctrl+i for inverting it, and you are done.
The effect is too strong? Lower the opacity in the slider above the layer and reduce the strength of the brightness change.
The technique of masking with channels is very sophisticated, but it does not work for all images and tasks. Take your time to test it with several images and you will learn to judge even before starting the process. But once this technique works it is extremely precise.
Ready for practicing – a lot!