Your logic seems to suggest that you want something more than just a decrement counter. A simple decrement counter would work exactly the same as an increment counter:
This logic will count down by one. To set a starting value, set the initial condition in the memory block to the desired starting value + 1. So, to start the counter from 20, set the memory initial condition to 21, and the first output from this network is 21-1 = 20.
If you want the counter to stop and hold a value, zero for example, simply add a saturation block:
Set the upper limit in the saturation block to a very large value, and set the lower limit to the desired floor value (0 maybe). This will count down from the initial value in the memory block and hold the lower limit value from the saturation block.