Preheat oven to 350°F (175°C). Grease a 9x13-inch baking dish with cooking spray.
Whisk flour, sugar, baking powder, and salt; stir in milk, eggs, melted butter, and vanilla until smooth.
Spread batter evenly into prepared baking dish.
Mix softened butter, brown sugar, cinnamon, and 2 tablespoons flour in a bowl until smooth paste forms.
Drop spoonfuls of cinnamon mixture over batter; swirl throughout using a butter knife.
Bake 28 to 30 minutes until a toothpick inserted in center comes out clean.
Whisk powdered sugar with milk and vanilla; drizzle warm glaze over cake immediately after baking.