Sounds like you have an issue with the VSSB (Vehicle Speed Sensor Buffer) or the VSS. The VSS is located on the rear housing of the tranny (the rear ABS isn't biased). I don't know where the VSSB is on a van, but likely in the dash somewhere. A poor ground (or any poor connection, for that matter) can cause what you are seeing.
(advice)
Tie a string to the ABS fuse and tape/tie it where you can reach it, so you can pull it in a hurry when your speedo goes to zero. Could be helpful if you have to brake unexpectedly. At least until you get the issue squared away.