Line your jelly roll pan with parchment paper and lightly grease it.
In a medium bowl, whisk together the all-purpose flour, baking powder, baking soda, salt, cinnamon, nutmeg, and ginger.
In a large bowl, use an electric mixer to beat the eggs and granulated sugar on high speed until thick and pale, about 5 minutes.
Beat in the pumpkin puree and 1 teaspoon of vanilla extract until well combined.
Gradually fold the dry ingredients into the wet mixture using a spatula, being careful not to overmix.
Pour the batter evenly into the prepared pan and smooth the top. Bake at 350°F (175°C) for 15 to 18 minutes, or until the cake springs back when lightly touched.
While the cake is still warm, carefully invert it onto a clean kitchen towel dusted with powdered sugar. Peel off the parchment paper and gently roll the cake with the towel from the short end. Let it cool completely in the rolled position.
In a medium bowl, beat the softened cream cheese and butter until smooth. Add powdered sugar, 1 teaspoon vanilla extract, and milk. Beat until fluffy and creamy.
Unroll the cooled cake carefully and spread the cream cheese filling evenly over the surface. Sprinkle with chopped walnuts or pecans if using. Re-roll the cake without the towel, wrapping it tightly.
Wrap the rolled cake in plastic wrap and refrigerate for at least one hour to set. Slice and serve chilled or at room temperature.