It's good that you bought a whole new valve. Best to rebuild it.
That valve is a Weathermatic Silver Bullet. I rebuild those all the time. They are known to stick open. I've found that the diaphragm ports, on older diaphragms especially, will swell shut causing it to stick open. Sometimes debris causes this too. Some of those valves will also start to scream like a banshee trying to close when they start to fail.
You could also use an eighteen gauge copper wire and push it through the two ports on the diaphragm to give it a little more life in a pinch, but it would fail again eventually. Go ahead be ready for the other's to fail too.
I know you've already taken the valve apart, so you noticed this already, but it may help someone else. When the valve's apart, be careful to notice the o-ring on the solenoid port or in the valve body. The new valve will have an o-ring too. Only use one o-ring. Use the new one. Also make sure the manual on/off lever is horizontal (off), not perpendicular (on) to the valve. Have fun!