What is Bracketing?
Let's start with a definition and then let's look at a few examples.
A bracket is "the filled-in area that connects the serif to a stroke ..." -Doyald Young
In contrast, take a look at what modern fonts do. They remove the brackets so the end of each stroke meets a serif at an angle, creating a sharp and clean aesthetic.
Another example of where bracketing can be used is on the ball terminal of a letter. Check out these lowercase letter 'a's for instance. Another thing to keep in mind is that, brackets are not one size fits all. You can change the size of a bracket to create a smooth transition like the ball terminal of the 'a' in Didot. Or you could have a smaller and tighter bracket, as seen at the inside juncture of the ball terminal on the 'a' of Bodoni. Palatino is an example of no brackets for the ball terminal.
Why use bracketing?
Great question! A functional reason why you might choose to include a bracket on a letter is if you have thin strokes and serifs. Adding brackets to a thin stroke and its serif can add a little bit of visual weight, to help balance out the letter. Check out this example of the lowercase letter 'y' in the font Bauer Bodoni. The top left serif doesn't need brackets because it is attached to a thick stroke, but the thin stroke on the top right does have brackets.