I just went through this with my 96. If you look around the net you will see this is a reported issue with this vintage of ABS. I just about got killed. My light would be off and then I go to stop and it would go on as I applied the brake. Same as you, getting a real hard pedal and a no stop situation. On mine I'm guessing it is a fouled up wheel sensor. I chose not to mess with it and disconnected the 60 amp ABS fuse under the hood. The ABS light is always on now, but that is far better than not being able to stop when you need to.
1996 K2500 Suburban
Second owner, Former Fire Dept Vehicle
"F" Engine, 4:10's looking for 3:73's
D-TECH Remote mounted PMD, HO water pump
1998 air box with K&N filter
Fluidampr
17" GMC Alloy Wheels with 265 70 R17's
RACOR 490 fuel filter with heater and 2 Micron Filter