Ndtyhij

I have this code that reads a file and after processing a few lines writes the output to a second file:

num_reads = 7

with open('data.txt') as read_file:

    with open('new_data.txt', 'w') as write_file:



        while (True):

            lines = 

            try:       # expect errors if the number of lines in the file are not a multiplication of num_reads

                for i in range(num_reads):     

                    lines.append(next(read_file))  # when the file finishes an exception occurs here



                #do sutff with the lines (exactly num_reads number of lines)

                processed = " ".join(list(map(lambda x: x.replace("n", ''), lines)))

                write_file.write(processed + 'n')



            except StopIteration:     # here we process the (possibly) insufficent last lines

                #do stuff with the lines (less that  num_reads)

                processed = " ".join(list(map(lambda x: x.replace("n", ''), lines)))

                write_file.write(processed + 'n')

                break

Here is the input file (data.txt):

line1

line2

line3

line4

line5

line7

line8

line9

And this is the output file that has the desired state:

line1 line2 line3 line4 line5 line7

line8 line9

This works correctly but as I wish to do the same processing and writing procedure in both cases (when the number of elements is 7 and when the file finishes and the exception is raised) I think the above code violates DRY principle even if I define a new function and call it once in try block and once in except before break. Any other ordering that I could come up with was either causing an infinite loop or losing the final lines.
I appreciate any comments on handling this issue, as it is not limited to this case and I had faced it in other cases as well.

Disclaimer: This question belongs to Stack Overflow, and I voted to migrate it. Therefore, the answer is not a review.

Keep in mind that principles are there to guide you. They should be treated like guard rails, rather than roadblocks.

I would argue that

    while (....) {

        foo(7);

    }

    foo(3);

does not violate DRY. Your situation is pretty much the same.

That said, your idea of defining function is valid. You just factoring out the wrong code. Factor out reading. Consider

    def read_n_lines(infile, n):

        lines = 

        try:

            for _ in range(n):

                lines.append(next(infile))

        except StopIteration:

            pass

        return lines

and use it as

    while True:

        lines = read_n_lines(infile, 7)

        if len(lines) == 0:

            break

        process_lines(lines)

You should avoid writing code with exception-handling altogether. Usually, when you want to write a fancy loop in Python, the itertools module is your friend. In this case, I would take advantage of itertools.groupby() to form groups of lines, assisted by itertools.count() to provide the line numbers.

import itertools



def chunks(iterable, n):

    i = itertools.count()

    for _, group in itertools.groupby(iterable, lambda _: next(i) // n):

        yield group



with open('data.txt') as read_f, open('new_data.txt', 'w') as write_f:

    for group in chunks(read_f, 7):

        print(' '.join(line.rstrip() for line in group), file=write_f)

A few other minor changes:

• You only need one with block to open both files.


• 
  line.rstrip() is more convenient than lambda x: x.replace("n", '')
  


• 
  print(…, file=write_file) is slightly more elegant than write_file.write(… + 'n').