append data to the already existed .txt file horizontally

I want to append the data to my already existed .txt file, how can I do that.

My text file is like that:

A B C D

1 2 3 4

1 2 3 4

1 2 3 4

1 2 3 4

and I want to add the data

E F G H

5 6 7 8

5 6 7 8

5 6 7 8

5 6 7 8

so the whole text file should be something like that:

A B C D E F G H

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

Answer 1

[KSSV]

T1 = readtable('file1.txt') ;
T2 = readtable('file2.txt') ;
T = [T1 T2] ;
writetable(T,'file.txt')

[MakM]

I want to add the data in the already existed .txt file, like first open the text file and then add data, because my data is in thousands of rows, and creating two txt files and then merging them takes time.

[KSSV]

Did you try it?

[MakM]

Yes..

Answer 2

[Walter Roberson]

If your files are already in .txt files, and they are in regular columns like you show, then the method that @KSSV shows will work fine.

You might perhaps be hoping to not have to read and interpret the text files and format back as text on output and writing out new files. If so we can break that up into two parts:

1. can we avoid reading and writing the files?
2. can we avoid intepreting as numeric and then formatting back to text again?

The answer to (1) is that something has to read and write files. The internal structure of text files on all supported operating systems is as continuous streams of characters, all of one line, then a line terminator, then all of the next line, line terminator, and so on. The line terminators are plain characters themselves: just linefeed character for MacOS and Linux, and potentially on Windows as well; carriage return character followed by linefeed character for older Windows programs. The file systems do not keep track of how long each line is, and the file systems do not have any way of saying "when you read in this file, add this segment on the end of line #83". The only thing the file systems support for text files is continuous streams of characters with no structure by the file system (other than to keep track of the length of the file.)

Because of this, if you want to append ' Hello' on the end of line 1 of a text input file, the best way is to read the first line, append ' Hello' to it, write that out to a new output file, then read lines from the input file and write them to that output file.

Is there an alternative to writing to a new file like this? Yes, there is if the revised file will not be smaller than the original file: you can read the first line, and insert ' Hello' into an output buffer, then start a process of reading ahead and adding to the output buffer and writing out from the output buffer only as many characters as you just read in. For example if you read 20 characters then you would write out ' Hello' followed by the next 14 characters, leaving the last 6 characters in the output buffer for the next cycle. Eventually you get to the end of the input file, and you write out the rest of the output buffer. But... if something goes wrong with this update-in-place process then you have ruined the file and cannot get it back. And notice that the total reading and writing characters is the same as for the write-to-new-file case. With the necessary buffer management to be sure that you do not accidentally overwrite something, the operations can turn out slower.

Does the "something" that does the reading and writing of the files have to be MATLAB? Well, if you are using MacOS or Linux, you can call upon the system utility paste to do the work; see https://unix.stackexchange.com/questions/58384/how-to-implement-a-horizontal-cat

This leads to question (2), whether we can avoid interpreting as numeric and then refomatting again. The answer to that is Yes, if you are willing to do the text management discussed earlier, or if you can call paste or equivalent.

Are there other text ways to handle this? Yes,

L1 = readlines('file1.txt');
L2 = readlines('file2.txt');
newLines = strcat(L1, {' '}, L2);
writelines(newLines, 'OutputFileNameGoesHere.txt');

It is important that the ' ' be inside {} for the purposes of strcat. If you try to strcat(L1, ' ', L2) you will not get the right answer!

[MakM]

Thanks Walter for the detailed answer, the first matrix[A B C D (I showed in the question)] is the txt file and the second matrix is the one I want to append, by changing all the matrices to txt file and then merge can make alot of mess (too many txt files) because I will have more than 2 matrices at some point, so if for example, I have 100 matrices I want to merge, I have to make 100 txt file and then merge, so I want to have a txt file which I can append horizontally time to time without having too may txt files, so all the data can be occupied in the single file. Also, I am copying the database rows into txt file which has thousand of rows, so creating txt file for each column or the couple of columns will create too many files, how can I append data (like add new column in the txt file), whenever I have new column of values.

[Walter Roberson]

Have you considered using a .xlsx file? You can specify a Range to write data into and that range can be after the existing columns. .xlsx are internally implimented as .xml (text) files that are zip'd together, so text files would get rewritten, but you would not have to manage the process. (But it would be most efficient to bundle data together to append, rather than writing one column at a time.)

[MakM]

I have a database file which has 11272228 rows with 14 column, whenever I read it using a simple query, for example, select column1, column2 from table, MATLAB crashes and shows out of memory issues. Although I have 8gb ddr4 and 512 gb ssd, so I am getting one column at a time, clear that column data (dynamically) which I know is making the process slower, but in order to avoid the crash I have to do this. As far as .xlsx is concerned, it does not shows all the rows, like if I save the 11272228 rows, It can not save all the rows because of huge data.

[Walter Roberson]

You are right, xlsx can only handle a bit over 1 million rows.

Can your database software handle appending columns ?

[MakM]

Database save all the columns at the same time, what I am trying to do is to shift the database data into some kind of text files or any other format so the data extraction should be faster.