Serving is all about the toss, timing and rhythm. Hitting a first serve more than 100 miles per hour and then hitting a second serve 20-30 miles per hour slower makes it very difficult to time the motion and point of contact.
In addition, the variation in the speed often affects the toss because the rhythm of the motion changes, resulting in the ball being released sooner or later than normal. I have always been an advocate of hitting your second serve aggressively. Most players slow down their second serve a great deal with the idea that this will help them make the serve. Instead, try keeping the speed of your swing similar and adding spin.
Regardless of the type (kick, slice, or just brushing across the back of the ball), the spin will bring the ball down into the box and allow you to hit the serve with more consistent timing. Using similar racquet head speed, aim to the middle of the service box, and hit both serves with the intent of getting your opponent in trouble. This will likely decrease the number of double faults and allow you to be aggressive when the score is close at the end of the match.