File Exchange

image thumbnail

Textscantool

version 1.12.0.1 (12.6 MB) by

App to read large text files

4.52941
20 Ratings

13 Downloads

Updated

View License

Editor's Note: This file was selected as MATLAB Central Pick of the Week

Note: Does not work in release 2014a or later. For these releases use the built-in MATLAB "Import Tool" accessed by selecting "Import Data..." from the current folder window.
Textscantool is an interactive app to read large text files with delimited data. The tool lets you specify which columns to read and what data types to use. It can also generate the MATLAB code to carry out the same task programmatically. There is a video tutorial for this tool in the directory that comes with this submission. Unzip the files to the current directory or somewhere on your path.

Comments and Ratings (41)

Stuart McGarrity

What release are you using Andrea? This tool does not work with 14a or later. Try the import tool in MATLAB.

Andrea

Andrea (view profile)

Error using textscantool (line 145)
The name 'mouseclickedCallback' is not an accessible property for an
instance of class
'com.mathworks.widgets.spreadsheet.SpreadsheetScrollPane$NamedRowHeaderTable'.

kevin

kevin (view profile)

textscantool
??? Error using ==> textscan
Buffer overflow (bufsize = 4095) while reading string from
file (row 15936, field 2). Use 'bufsize' option. See HELP TEXTSCAN.

Error in ==> textscantool>importDataCallBack at 848
                dataCell=textscan(fid,formatString,numBlockRowstoRead,... %
                Read data
 
??? Error while evaluating uicontrol Callback

This is definitely a nice tool. However, the folks who generate some of the data files I have have thrown a few curve balls. While I can reprocess the data from strings, I don't think it would be hard for you to add a few new features.
1. A new class of logical which translates strings of f or false or 0 or off to logical false (any case) and t or true or 1 or on to logical true.
2. A class which takes a set of string values that have a limited set of possible values and translates them into numbers. For example, there might be three different status types, e.g., standby, off, and on. We might like to put in numbers such as 0 for standby, 1 for off, and 2 for on. It would be handy to have textscantool do that translation rather than have us do the conversion later.

Selcuk Fidan

Stuart McGarrity

There should be a "Count Lines in File" button at the end just before importing. Do you see it?

Roberto

Great file, it helped me a lot to read some messy data that I have, however, I need to figure out now how many rows does my files has, any thought or ideas to work this out?
Cheers.
PS.: Sorry for the blanks post before... my bad

Roberto

Roberto

Roberto

Mohamed

??? The class
com.mathworks.mlservices.MLEditorServices has no
property or method named 'newDocument'.

Error in ==> textscantool>generatefile at 638
        com.mathworks.mlservices.MLEditorServices.newDocument(str,true);
??? Error while evaluating uicontrol Callback

Ariel

Ariel (view profile)

Great work !!

Foroguh

I am getting an error:
Too many inputs.

Error in ==> textscantool at 84
pushButtonHandlesCell=cellfun(@(x) findobj(fig,'Tag',x),pushButtonHandlesGUITags,'Uniform',false);

can u help me plz?

William

With Matlab 2010b (7.11.0.584) get error when press 'Generate Code' button:

??? The class com.mathworks.mlservices.MLEditorServices has no property or method
named 'newDocument'.

Error in ==> textscantool>generatefile at 638
        com.mathworks.mlservices.MLEditorServices.newDocument(str,true);
 
??? Error while evaluating uicontrol Callback

Otherwise tool worked for me, thanks!

Braden

Braden (view profile)

I had luck using textscantool with the 1D row cell array but have ran into a snag attempting to use 2D cell array on the same data. This is the error I have picked up:

??? Subscripted assignment dimension mismatch.

Error in ==> textscantool>importDataCallBack at 852
                    data(2:end,k)=dataCell{k}; %.. just copy in
 
??? Error while evaluating uicontrol Callback

Let me know if you need more information to debug this.

burcu

burcu (view profile)

Thanks for the advice Stuart but now i have 2 new errors. Sorry not to mentioning it first but my version is 7.2.0.232, could it be the reason?

??? Operands to the || and && operators must be convertible to logical scalar values.

Error in ==> uitable at 170
            else if (isnumeric(varargin{i+1}) && size(varargin{i+1}) == [1 3])

Error in ==> textscantool at 127
    uitableJavaHandles.lineStringsTable=uitable('Parent',fig,'Data',1, 'ColumnNames',{''},'gridcolor',GUIcolor);

Stuart McGarrity

Don't add the code at the very begining. Add if just before the first "if str2double..." line you insert.

burcu

burcu (view profile)

Hi Stuart,

I was facing same issue with Eleni and applied your items and get another error:
??? Error: File: textscantool.m Line: 2 Column: 1
Function definitions are not permitted at the prompt or in scripts.
Do you have any comment on this?
Thanks!
Burcu

Stephen Lienhard

Eleni Vlachou

It worked!!!!!!!!!!!! Thank you so much. I really appreciate your help.

Eleni

Stuart McGarrity

Add this line at the beginning of textscantool.m
versionStr=version;

Then replace both instances of:
if verLessThan('MATLAB','7.6')

With:
if str2double(versionStr(1:3))<7.6

It should work in version 7.1

Eleni Vlachou

Hi,

I have 7.1 :-( Is there anything else similar to this? I was so happy I found this.

Thank you.

Eleni

Stuart McGarrity

What version of MATLAB do you have? This file only works for release 7a (MATLAB 7.4 ) or later.

Eleni Vlachou

Hi,

I tired using this, seems amazing and exactly what I need but I get this error:

??? Undefined command/function 'verLessThan'.

Error in ==> textscantool at 125
if verLessThan('MATLAB','7.6') % Before 8a
 I have matlab 7.1

Any thoughts? Thank you very much.

Eleni

Matteo

Matteo (view profile)

Yes Stuart I've checked and I also tried to download it again....I had the same problem on another PC as well...really strange ..I will try to investigate more on my side as well.

Stuart McGarrity

I can't reproduce this in 9a. Have you checked that you have the fig file? If so, have you tried downloading again?

Matteo

Matteo (view profile)

did anyone experience an error using this fantastic tool on Matlab 2009a? I have this problem when launching it:
Warning: an error occurred while parsing class uitools.uibuttongroup:
Attempt to reference field of non-structure array.
 
> In hgload at 43
  In openfig at 72
  In textscantool at 66
??? Attempt to reference field of non-structure array.

Farah

Farah (view profile)

My problem above was easily fixed, I had not entered in the right key for tab delimited, which is '\t'. Stuart pointed this out to me :). Textscantool worked super fast after, opening what it said would take an hour in a second

Farah

Farah (view profile)

Hi, this is just what I needed!!!! thanks so much for uploading. I'm working with microarray data (2.1 million rows and multiple column .txt files) and need to selectively import rows, so it's perfect.
But I have encountered a problem. My .txt files are tab delimited, however textscantool reads it as comma delimited. I tried changing the delimiter manually substituting '|' for ',' but that didn't help. Can you please let me know how to fix this? thanks!

Gene Leynes

OMG, this thing is the cat's meow. I can't believe Mathwork's doesn't have something like this.
Some ideas:
+ Read the first few row(s) of data to test columns for number values for smarter defaults (rather than all strings). I wrote something like this in VBA if you’d like it.
+ Include brief data definitions for the data types (do I need an int32 or int64? I don’t know!)

J Kim

>> textscantool
??? Too many inputs.

Error in ==> textscantool at 82
pushButtonHandlesCell=cellfun(@(x) findobj(fig,'Tag',x),pushButtonHandlesGUITags,'Uniform',false);

Could you please let me know what's wrong?

best matlabber

too sweet!

Max J

generated code runs real quick

w s

works like a charm. Thanks a lot!

Vick Engineer

Love the tool. Wish the Import Wizard had similar functionality.

Enhancement request:
When bringing in data that contains one column of dates and the rest numbers, upon selecting the column that contains dates and then selecting the radio button to apply the datenum function, the entire data cannot be brought in a 2D numerical array. I would think that after applying the datenum function all columns are now of the same type.

A K

Can't make this work. Any suggestions? These are the error.

??? Error while evaluating uicontrol Callback.

??? No method 'newDocument' with matching signature found for class 'com.mathworks.mlservices.MLEditorServices'.

L Li

bug in line 82

Marcelo Perlin

Tks Stuart. Excelent tool.

steohen hope

it's really a usefull tool for import data!

Matteo Sorci

Excellent tool. I was looking or something similar for a while and finally it appears! Thanks a lot!

Updates

1.12.0.1

Updated license

1.12

Added note about supported MATLAB release.

1.11

Fixed errors and warings in release 10b and 11a.

1.7

Removed gif.

1.5

Copyright work.

1.3

Typo work.

1.2

Fixed bug in line count (sometimes off by 1). Fixed example date format to a valid one. Thanks to Michael Weidman for these.

Fixed bug causing error with short files.

Changed name of generated function to 'dataimport' to avoid clashing with existing 'importdata' function

Try upload again

MATLAB Release
MATLAB 7.4 (R2007a)

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video