1) You can mark the column as editable. Use a CellEditCallback to examine the user's input after they have pressed Return / Enter or clicked elsewhere, but before it has been saved, and modify the data before it is saved. The available information includes which cell is being edited. However, this does not work on a character-by-character basis.
2) You can use mark the column as not editable. Use CellSelectionCallback to take a record of which cell the user clicked on. Use KeyPressFcn callback to see which keys the user is typing, and have it selectively make changes to the content of the appropriate cell (including handling backspace for editing.) KeyPressFcn does not give information about where in the table the user clicked. This cannot be said to be a convenient way to handle the task. I do not recall at the moment what happens if you have a KeyPressFcn active on an editable cell: something similar is possible with uicontrol style 'edit', and that combination has oddities about when the characters stored by KeyPressFcn become available.
3) You can head over to http://undocumentedmatlab.com and read about the java internals of uitables and hope to find a suitable edit mechanism described there.