Here are two different ways to create artistic or rough photo edges.
- Open your image in Photoshop.
- Make a rectangular selection and click on the add layer mask button at the bottom of the layers palette.
- Click on the layer mask (important!) and apply Filter > Brush Strokes > Spatter. For this example I used the values spray radius: 18, smoothness: 11.
- To smooth the bitmapped edge, go to Select > Refine Edge. My values for this example were radius: 1, contrast: 20%, smooth: 3, feather: 0, contract: 0.
- Draw a rectangle on a blank page in Illustrator.
- Add a black fill and 1.5 pt black stroke. Apply Effect > Stylize > Round Corners and use amount: 2 pt.
- Open the Artistic_ChalkCharcoalPencil brush library and apply the Chalk - Scribble brush to the stroke.
- Select the rectangle, copy it and paste a copy in front (Cmd-F). Rotate the copied rectangle 180 degrees. (This step fills in any empty corners. Often a brush stroke will get too thin or disappear in one corner.)
- Select all (Cmd-A) and choose Object > Expand Appearance. Select all again and copy the rectangle to the clipboard.
- Switch to Photoshop and open your image file.
- Paste the rectangle as a shape layer with a black fill. Scale up or down as necessary.
- On the shape layer, Cmd-click on the vector mask to load a selection of that mask.
- On the photo layer, click the add layer mask button at the bottom of the layers palette. Hide the shape layer.