Compare date & time (efficiency question)

Hello everyone

Quick 2 questions:

Background: I did import lots of data using "textscan" where times and dates are crucial biases for analysis (certain actions happened at certain times on certain dates) therefor I need to pick and choose data relative to certain times & dates (i.e. if time of dataA is less than 10:00:00, do such & such). I know I can convert dates & times using “datenum” & “datestr”, then do whatever comparison I need & execute actions.

My questions are:

1) Are there any other code method that I can use to compare dates & time instead of the above codes/method?

2) In terms of computing efficiency (speed, memory, etc.), is it better if the pick & choose dates & times that I need to look at while I am importing the data using “textscan”? or shall I stick to what I am doing of importing times & dates as strings, convert them into numerical, compare & actions, then convert them into strings (if I need to)?

Thanks

Tariq

Answer 1

[per isakson]

• "lots of data" means different things to different people. Does it fit in memory - installed RAM? RAM is cheap.
• datenum converts to serial data number, sdn. Together with ">" and "<" sdn is good for "before" and "after" type questions. Not so good for exact comparisons (float rounding error), exactly 12:00:00 - requires a of bit care.
• datestr is good for output and presentation. Period. Good for people to read.
• datevec converts to - lets call it date_vec. date_vec is good for question like between 16:00 and 17:59:59 every day. Must be combined with holidays.
• Q1: datevec, weekday, ... (Finance Toolbox should be full of useful stuff.)
• Q2: Textfiles are painful to work with! You tell us too little about what you want to do. "computing efficiency" is not an end to itself. Use the ram and let the cpu work. Is it about GigaBytes of data at your "fingertips"? Diagram shall display before the Return key springs back? Or do you run batch jobs all night?

My recipe

1. read textfiles once
2. convert data and to an appropriate form. Describe a couple (or more) of typical request you will make.
3. store it in a better place - have a look at the function, matfile. However, we want a version that is more capable with indexing. Not too many cell arrays - ideally zero.

BTW: I have started to use serial second number, ssn, which a store as integers. Stock exchange work with microseconds - or what?

[Tariq]

Thanks for your answer

I am already using what you have mentioned in your answer. There are lots of things going on in my files of data & the analysis.

I will come back to you with much more details about my problem (and data) if I can't do any progress.

Thanks again

Tariq