The Cardinals improved the major league's best record to 32-17 and increased their NL Central lead over Cincinnati to 1 1/2 games.
Kozma led off the seventh with a bloop double off the glove of shortstop Dee Gordon as he dived for the ball in short center field. One out later, Carpenter grounded a single off the glove of first baseman Adrian Gonzalez, allowing Kozma to score. Pinch-hitter Matt Adams added an RBI single in the ninth.
Major league ERA leader Kershaw had a chance to help himself with the bat in the sixth, but grounded into an inning-ending double play.
Edward Mujica pitched a perfect ninth inning for his 14th save in 14 attempts.
Kershaw (5-3) gave up four runs, seven hits and three walks in seven innings and struck out five as his ERA rose from 1.35 to 1.68. The 2010 Cy Young Award winner had allowed only three earned runs in 39 2-3 innings over his previous five starts, including two no-decisions.
Kershaw had not allowed more than three earned runs in any of his previous 22 starts since giving up eight over 5 2-3 innings against the Cardinals on July 24, 2012 at St. Louis. That was the longest such streak by a Dodger since Orel Hershiser's 27-game stretch between July 12, 1985 and May 25, 1986.
Slumping center fielder Matt Kemp was not in the Dodgers' starting lineup, a day after he was removed by manager Don Mattingly in a double-switch with the team ahead by a run in the seventh inning of a 5-3 win. The two-time All-Star pinch-hit for reliever Ronald Belisario in the eighth with a man on and grounded into an inning-ending double play.
The Dodgers loaded the bases in the bottom of the inning after Trevor Rosenthal followed Maness out of the bullpen and hit his first batter, Andre Ethier. But Rosenthal escaped the jam by striking out A.J. Ellis and Skip Schumaker, who started in center field in Kemp's place.
Miller allowed three runs and seven hits in 5 2-3 innings and struck out three in his first career start against the Dodgers, after beginning the day with the fourth-best ERA in the NL at 1.74.
The Dodgers took a 2-0 lead in the first when Gonzalez drove a 1-1 pitch into the right field pavilion for his team-high sixth home run, after a leadoff walk to Carl Crawford.
But the Cardinals countered with three runs in the third. Yadier Molina drew a leadoff walk, David Freese hit a broken-bat bloop double down the left field line on the next pitch, and .229-hitting Daniel Descalso walked on four pitches. Kozma then had his bases-clearing double just inside third base and past a diving Juan Uribe.
Los Angeles tied it in the fifth on an RBI single by Gonzalez that scored Kershaw, who started the rally with a one-out single.
NOTES: Ellis has struck out seven times in nine at-bats since his collision at home plate with Cardinals baserunner Jon Jay in Friday night's series opener. In Ellis' previous 19 at-bats, he went down on strikes only once. ... Kemp is homerless in 101 at-bats at Dodger Stadium since his two-run shot off Colorado's Jorge De La Rosa last Sept. 20. ... Cardinals LHP John Gast, who left Saturday's game with tightness in his shoulder after facing just six batters, was placed on the 15-day disabled list - making him the fourth Cardinals starter currently on the DL along with Chris Carpenter, Jake Westbrook and Jaime Garcia. ... RHP Michael Blazek, another rookie, was promoted to the majors for the first time from Triple-A Springfield. Blazek will be the seventh rookie pitcher used this season by manager Mike Matheny. ... St. Louis RF Carlos Beltran, 4 for 19 with seven strikeouts against Kershaw, got the day off. ... The Dodgers play the first of four straight games against the Angels on Monday night - the first two at Chavez Ravine.
Link to box score: St. Louis 5 at Los Angeles 3
Link to game discussion at The Cardinal Nation message board