Preheat the oven to 350˚F and prepare three 8-inch or four 6-inch cake pans by spraying the sides with cooking spray and fitting a parchment or wax paper circle to the bottom of each pan.
Grind the graham crackers into a crumbs by either using a food processor or placing them into a large ziplock back and crushing them with a rolling pin. Place the graham cracker crumbs into a medium sized bowl and add the sifted cake flour, baking powder, baking soda, and salt. Whisk to combine and set aside.
In the bowl of a stand mixer with the paddle attachment, cream butter for one minute on high. Scrape down the bowl and paddle and add the sugar. Cream together on high for two minutes, scraping down bowl and paddle once in between. Turn mixer to low and add the eggs, one at a time, scraping down bowl and paddle as necessary. Once all eggs are added, turn the mixer to high and beat for three minutes.
Add vanilla and sour cream and continue mixing for one minute. Turn the mixer to low and add all of the dry ingredients at once until just combined, then add the milk all at once. Scrape down the bowl and paddle once more and mix on low for about 30 seconds. Do not overmix. Pour batter into prepared pans (about ⅔ full). Bake for 35-45 minutes or until a toothpick inserted into the center comes out clean. Cool the cakes for five minutes before releasing from the pan and continuing to cool on a baking sheet or wire rack.