Unexpected behaviour with Python generator












13















I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.



I tested this simple code:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original to something else

print(list(f)) # Outputs filtered


And the output was:



>>> 


Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn't get that:



# Expected output
>>> [2, 2]


When I commented out the third line to test it once again:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
### array = [5, 6, 1, 2, 9] # Ignore line

print(list(f)) # Outputs filtered


The output was correct (you can test it for yourself):



>>> [2, 2]


At one point I outputted the type of the variable 'f':



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original

print(type(f))
print(list(f)) # Outputs filtered


And I got:



>>> <class 'generator'>
>>>


TL;DR: Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.










share|improve this question




















  • 2





    You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

    – Prune
    7 hours ago











  • Yh that is what I meant sorry

    – Suraj Kothari
    7 hours ago






  • 3





    You redefine array and your new array is what gets referenced by the lazy generator comprehension.

    – jpp
    7 hours ago













  • Would be good to see an answer that mentions scope.

    – trailing_whitespace
    40 mins ago











  • Slap "strange" onto the title and you're sure to get a few upvotes.

    – coldspeed
    34 mins ago


















13















I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.



I tested this simple code:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original to something else

print(list(f)) # Outputs filtered


And the output was:



>>> 


Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn't get that:



# Expected output
>>> [2, 2]


When I commented out the third line to test it once again:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
### array = [5, 6, 1, 2, 9] # Ignore line

print(list(f)) # Outputs filtered


The output was correct (you can test it for yourself):



>>> [2, 2]


At one point I outputted the type of the variable 'f':



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original

print(type(f))
print(list(f)) # Outputs filtered


And I got:



>>> <class 'generator'>
>>>


TL;DR: Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.










share|improve this question




















  • 2





    You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

    – Prune
    7 hours ago











  • Yh that is what I meant sorry

    – Suraj Kothari
    7 hours ago






  • 3





    You redefine array and your new array is what gets referenced by the lazy generator comprehension.

    – jpp
    7 hours ago













  • Would be good to see an answer that mentions scope.

    – trailing_whitespace
    40 mins ago











  • Slap "strange" onto the title and you're sure to get a few upvotes.

    – coldspeed
    34 mins ago
















13












13








13








I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.



I tested this simple code:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original to something else

print(list(f)) # Outputs filtered


And the output was:



>>> 


Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn't get that:



# Expected output
>>> [2, 2]


When I commented out the third line to test it once again:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
### array = [5, 6, 1, 2, 9] # Ignore line

print(list(f)) # Outputs filtered


The output was correct (you can test it for yourself):



>>> [2, 2]


At one point I outputted the type of the variable 'f':



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original

print(type(f))
print(list(f)) # Outputs filtered


And I got:



>>> <class 'generator'>
>>>


TL;DR: Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.










share|improve this question
















I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.



I tested this simple code:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original to something else

print(list(f)) # Outputs filtered


And the output was:



>>> 


Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn't get that:



# Expected output
>>> [2, 2]


When I commented out the third line to test it once again:



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
### array = [5, 6, 1, 2, 9] # Ignore line

print(list(f)) # Outputs filtered


The output was correct (you can test it for yourself):



>>> [2, 2]


At one point I outputted the type of the variable 'f':



array = [1, 2, 2, 4, 5] # Original array
f = (x for x in array if array.count(x) == 2) # Filters original
array = [5, 6, 1, 2, 9] # Updates original

print(type(f))
print(list(f)) # Outputs filtered


And I got:



>>> <class 'generator'>
>>>


TL;DR: Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.







python






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 11 mins ago









Ian Kemp

16.5k126798




16.5k126798










asked 7 hours ago









Suraj KothariSuraj Kothari

468213




468213








  • 2





    You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

    – Prune
    7 hours ago











  • Yh that is what I meant sorry

    – Suraj Kothari
    7 hours ago






  • 3





    You redefine array and your new array is what gets referenced by the lazy generator comprehension.

    – jpp
    7 hours ago













  • Would be good to see an answer that mentions scope.

    – trailing_whitespace
    40 mins ago











  • Slap "strange" onto the title and you're sure to get a few upvotes.

    – coldspeed
    34 mins ago
















  • 2





    You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

    – Prune
    7 hours ago











  • Yh that is what I meant sorry

    – Suraj Kothari
    7 hours ago






  • 3





    You redefine array and your new array is what gets referenced by the lazy generator comprehension.

    – jpp
    7 hours ago













  • Would be good to see an answer that mentions scope.

    – trailing_whitespace
    40 mins ago











  • Slap "strange" onto the title and you're sure to get a few upvotes.

    – coldspeed
    34 mins ago










2




2





You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

– Prune
7 hours ago





You keep printing list(g), but g is an undefined symbol. Perhaps you have a typo, using g for f?

– Prune
7 hours ago













Yh that is what I meant sorry

– Suraj Kothari
7 hours ago





Yh that is what I meant sorry

– Suraj Kothari
7 hours ago




3




3





You redefine array and your new array is what gets referenced by the lazy generator comprehension.

– jpp
7 hours ago







You redefine array and your new array is what gets referenced by the lazy generator comprehension.

– jpp
7 hours ago















Would be good to see an answer that mentions scope.

– trailing_whitespace
40 mins ago





Would be good to see an answer that mentions scope.

– trailing_whitespace
40 mins ago













Slap "strange" onto the title and you're sure to get a few upvotes.

– coldspeed
34 mins ago







Slap "strange" onto the title and you're sure to get a few upvotes.

– coldspeed
34 mins ago














6 Answers
6






active

oldest

votes


















9














As others have mentioned Python generators are lazy. When this line is run:



f = (x for x in array if array.count(x) == 2) # Filters original


nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call



print(list(f)) # Outputs filtered


the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.



If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:



f = [x for x in array if array.count(x) == 2] # Filters original
...
print(f)





share|improve this answer

































    3














    Generators are lazy, they won't be evaluated until you iterate through them. In this case that's at the point you create the list with the generator as input, at the print.






    share|improve this answer


























    • When am I iterating through them. Am I meant to?

      – Suraj Kothari
      7 hours ago











    • @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

      – Mark Ransom
      7 hours ago











    • Also which list? When I declare the first one, or re-assign the second?

      – Suraj Kothari
      7 hours ago











    • What first & second? You define only one list, at the final line of your code.

      – Prune
      7 hours ago











    • @SurajKothari I updated the answer to be more clear.

      – Mark Ransom
      7 hours ago



















    2














    You are not using a generator correctly if this is the primary use of this code. Use a list comprehension instead of a generator comprehension. Just replace the parentheses with brackets. It evaluates to a list if you don't know.



    array = [1, 2, 2, 4, 5]
    f = [x for x in array if array.count(x) == 2]
    array = [5, 6, 1, 2, 9]

    print(f)
    #[2, 2]


    You are getting this response because of the nature of a generator. You're calling the generator when it't contents will evaluate to






    share|improve this answer


























    • Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

      – Suraj Kothari
      7 hours ago











    • With your change, list(f) becomes redundant.

      – Mark Ransom
      7 hours ago











    • Lol @Mark Ransom, copy paste got me, I edited.

      – Jaba
      7 hours ago













    • @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

      – Jaba
      7 hours ago



















    0














    Generator evaluation is "lazy" -- it doesn't get executed until you actualize it with a proper reference. With your line:



    Look again at your output with the type of f: that object is a generator, not a sequence. It's waiting to be used, an iterator of sorts.



    Your generator isn't evaluated until you start requiring values from it. At that point, it uses the available values at that point, not the point at which it was defined.





    Code to "make it work"



    That depends on what you mean by "make it work". If you want f to be a filtered list, then use a list, not a generator:



    f = [x for x in array if array.count(x) == 2] # Filters original





    share|improve this answer


























    • I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

      – Suraj Kothari
      7 hours ago



















    0














    Generators are lazy and your newly defined array is used when you exhaust your generator after redefining. Therefore, the output is correct. A quick fix is to use a list comprehension by replacing parentheses () by brackets .



    Moving on to how better to write your logic, counting a value in a loop has quadratic complexity. For an algorithm that works in linear time, you can use collections.Counter to count values, and keep a copy of your original list:



    from collections import Counter

    array = [1, 2, 2, 4, 5] # original array
    counts = Counter(array) # count each value in array
    old_array = array.copy() # make copy
    array = [5, 6, 1, 2, 9] # updates array

    # order relevant
    res = [x for x in old_array if counts[x] >= 2]
    print(res)
    # [2, 2]

    # order irrelevant
    from itertools import chain
    res = list(chain.from_iterable([x]*count for x, count in counts.items() if count >= 2))
    print(res)
    # [2, 2]


    Notice the second version doesn't even require old_array and is useful if there is no need to maintain ordering of values in your original array.






    share|improve this answer































      0














      The root cause of the problem is that generators are lazy; variables are evaluated each time:



      >>> l = [1, 2, 2, 4, 5, 5, 5]
      >>> filtered = (x for x in l if l.count(x) == 2)
      >>> l = [1, 2, 4, 4, 5, 6, 6]
      >>> list(filtered)
      [4]


      It iterates over the original list and evaluates the condition with the current list. In this case, 4 appeared twice in the new list, causing it to appear in the result. It only appears once in the result because it only appeared once in the original list. The 6s appear twice in the new list, but never appear in the old list and are hence never shown.



      Full function introspection for the curious (the line with the comment is the important line):



      >>> l = [1, 2, 2, 4, 5]
      >>> filtered = (x for x in l if l.count(x) == 2)
      >>> l = [1, 2, 4, 4, 5, 6, 6]
      >>> list(filtered)
      [4]
      >>> def f(original, new, count):
      current = original
      filtered = (x for x in current if current.count(x) == count)
      current = new
      return list(filtered)

      >>> from dis import dis
      >>> dis(f)
      2 0 LOAD_FAST 0 (original)
      3 STORE_DEREF 1 (current)

      3 6 LOAD_CLOSURE 0 (count)
      9 LOAD_CLOSURE 1 (current)
      12 BUILD_TUPLE 2
      15 LOAD_CONST 1 (<code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>)
      18 LOAD_CONST 2 ('f.<locals>.<genexpr>')
      21 MAKE_CLOSURE 0
      24 LOAD_DEREF 1 (current)
      27 GET_ITER
      28 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
      31 STORE_FAST 3 (filtered)

      4 34 LOAD_FAST 1 (new)
      37 STORE_DEREF 1 (current)

      5 40 LOAD_GLOBAL 0 (list)
      43 LOAD_FAST 3 (filtered)
      46 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
      49 RETURN_VALUE
      >>> f.__code__.co_varnames
      ('original', 'new', 'count', 'filtered')
      >>> f.__code__.co_cellvars
      ('count', 'current')
      >>> f.__code__.co_consts
      (None, <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>, 'f.<locals>.<genexpr>')
      >>> f.__code__.co_consts[1]
      <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>
      >>> dis(f.__code__.co_consts[1])
      3 0 LOAD_FAST 0 (.0)
      >> 3 FOR_ITER 32 (to 38)
      6 STORE_FAST 1 (x)
      9 LOAD_DEREF 1 (current) # This loads the current list every time, as opposed to loading a constant.
      12 LOAD_ATTR 0 (count)
      15 LOAD_FAST 1 (x)
      18 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
      21 LOAD_DEREF 0 (count)
      24 COMPARE_OP 2 (==)
      27 POP_JUMP_IF_FALSE 3
      30 LOAD_FAST 1 (x)
      33 YIELD_VALUE
      34 POP_TOP
      35 JUMP_ABSOLUTE 3
      >> 38 LOAD_CONST 0 (None)
      41 RETURN_VALUE
      >>> f.__code__.co_consts[1].co_consts
      (None,)


      To reiterate: The list to be iterated is only loaded once. Any closures in the condition or expression, however, are loaded from the enclosing scope each iteration. They are not stored in a constant.



      The best solution for your problem would be to create a new variable referencing the original list and use that in your generator expression,.






      share|improve this answer

























        Your Answer






        StackExchange.ifUsing("editor", function () {
        StackExchange.using("externalEditor", function () {
        StackExchange.using("snippets", function () {
        StackExchange.snippets.init();
        });
        });
        }, "code-snippets");

        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "1"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54245618%2funexpected-behaviour-with-python-generator%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        6 Answers
        6






        active

        oldest

        votes








        6 Answers
        6






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        9














        As others have mentioned Python generators are lazy. When this line is run:



        f = (x for x in array if array.count(x) == 2) # Filters original


        nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call



        print(list(f)) # Outputs filtered


        the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.



        If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:



        f = [x for x in array if array.count(x) == 2] # Filters original
        ...
        print(f)





        share|improve this answer






























          9














          As others have mentioned Python generators are lazy. When this line is run:



          f = (x for x in array if array.count(x) == 2) # Filters original


          nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call



          print(list(f)) # Outputs filtered


          the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.



          If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:



          f = [x for x in array if array.count(x) == 2] # Filters original
          ...
          print(f)





          share|improve this answer




























            9












            9








            9







            As others have mentioned Python generators are lazy. When this line is run:



            f = (x for x in array if array.count(x) == 2) # Filters original


            nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call



            print(list(f)) # Outputs filtered


            the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.



            If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:



            f = [x for x in array if array.count(x) == 2] # Filters original
            ...
            print(f)





            share|improve this answer















            As others have mentioned Python generators are lazy. When this line is run:



            f = (x for x in array if array.count(x) == 2) # Filters original


            nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call



            print(list(f)) # Outputs filtered


            the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.



            If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:



            f = [x for x in array if array.count(x) == 2] # Filters original
            ...
            print(f)






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited 7 hours ago

























            answered 7 hours ago









            StevenSteven

            1067




            1067

























                3














                Generators are lazy, they won't be evaluated until you iterate through them. In this case that's at the point you create the list with the generator as input, at the print.






                share|improve this answer


























                • When am I iterating through them. Am I meant to?

                  – Suraj Kothari
                  7 hours ago











                • @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                  – Mark Ransom
                  7 hours ago











                • Also which list? When I declare the first one, or re-assign the second?

                  – Suraj Kothari
                  7 hours ago











                • What first & second? You define only one list, at the final line of your code.

                  – Prune
                  7 hours ago











                • @SurajKothari I updated the answer to be more clear.

                  – Mark Ransom
                  7 hours ago
















                3














                Generators are lazy, they won't be evaluated until you iterate through them. In this case that's at the point you create the list with the generator as input, at the print.






                share|improve this answer


























                • When am I iterating through them. Am I meant to?

                  – Suraj Kothari
                  7 hours ago











                • @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                  – Mark Ransom
                  7 hours ago











                • Also which list? When I declare the first one, or re-assign the second?

                  – Suraj Kothari
                  7 hours ago











                • What first & second? You define only one list, at the final line of your code.

                  – Prune
                  7 hours ago











                • @SurajKothari I updated the answer to be more clear.

                  – Mark Ransom
                  7 hours ago














                3












                3








                3







                Generators are lazy, they won't be evaluated until you iterate through them. In this case that's at the point you create the list with the generator as input, at the print.






                share|improve this answer















                Generators are lazy, they won't be evaluated until you iterate through them. In this case that's at the point you create the list with the generator as input, at the print.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 7 hours ago

























                answered 7 hours ago









                Mark RansomMark Ransom

                223k29281508




                223k29281508













                • When am I iterating through them. Am I meant to?

                  – Suraj Kothari
                  7 hours ago











                • @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                  – Mark Ransom
                  7 hours ago











                • Also which list? When I declare the first one, or re-assign the second?

                  – Suraj Kothari
                  7 hours ago











                • What first & second? You define only one list, at the final line of your code.

                  – Prune
                  7 hours ago











                • @SurajKothari I updated the answer to be more clear.

                  – Mark Ransom
                  7 hours ago



















                • When am I iterating through them. Am I meant to?

                  – Suraj Kothari
                  7 hours ago











                • @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                  – Mark Ransom
                  7 hours ago











                • Also which list? When I declare the first one, or re-assign the second?

                  – Suraj Kothari
                  7 hours ago











                • What first & second? You define only one list, at the final line of your code.

                  – Prune
                  7 hours ago











                • @SurajKothari I updated the answer to be more clear.

                  – Mark Ransom
                  7 hours ago

















                When am I iterating through them. Am I meant to?

                – Suraj Kothari
                7 hours ago





                When am I iterating through them. Am I meant to?

                – Suraj Kothari
                7 hours ago













                @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                – Mark Ransom
                7 hours ago





                @SurajKothari when you create the list it will iterate for you without you needing to do it explicitly.

                – Mark Ransom
                7 hours ago













                Also which list? When I declare the first one, or re-assign the second?

                – Suraj Kothari
                7 hours ago





                Also which list? When I declare the first one, or re-assign the second?

                – Suraj Kothari
                7 hours ago













                What first & second? You define only one list, at the final line of your code.

                – Prune
                7 hours ago





                What first & second? You define only one list, at the final line of your code.

                – Prune
                7 hours ago













                @SurajKothari I updated the answer to be more clear.

                – Mark Ransom
                7 hours ago





                @SurajKothari I updated the answer to be more clear.

                – Mark Ransom
                7 hours ago











                2














                You are not using a generator correctly if this is the primary use of this code. Use a list comprehension instead of a generator comprehension. Just replace the parentheses with brackets. It evaluates to a list if you don't know.



                array = [1, 2, 2, 4, 5]
                f = [x for x in array if array.count(x) == 2]
                array = [5, 6, 1, 2, 9]

                print(f)
                #[2, 2]


                You are getting this response because of the nature of a generator. You're calling the generator when it't contents will evaluate to






                share|improve this answer


























                • Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                  – Suraj Kothari
                  7 hours ago











                • With your change, list(f) becomes redundant.

                  – Mark Ransom
                  7 hours ago











                • Lol @Mark Ransom, copy paste got me, I edited.

                  – Jaba
                  7 hours ago













                • @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                  – Jaba
                  7 hours ago
















                2














                You are not using a generator correctly if this is the primary use of this code. Use a list comprehension instead of a generator comprehension. Just replace the parentheses with brackets. It evaluates to a list if you don't know.



                array = [1, 2, 2, 4, 5]
                f = [x for x in array if array.count(x) == 2]
                array = [5, 6, 1, 2, 9]

                print(f)
                #[2, 2]


                You are getting this response because of the nature of a generator. You're calling the generator when it't contents will evaluate to






                share|improve this answer


























                • Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                  – Suraj Kothari
                  7 hours ago











                • With your change, list(f) becomes redundant.

                  – Mark Ransom
                  7 hours ago











                • Lol @Mark Ransom, copy paste got me, I edited.

                  – Jaba
                  7 hours ago













                • @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                  – Jaba
                  7 hours ago














                2












                2








                2







                You are not using a generator correctly if this is the primary use of this code. Use a list comprehension instead of a generator comprehension. Just replace the parentheses with brackets. It evaluates to a list if you don't know.



                array = [1, 2, 2, 4, 5]
                f = [x for x in array if array.count(x) == 2]
                array = [5, 6, 1, 2, 9]

                print(f)
                #[2, 2]


                You are getting this response because of the nature of a generator. You're calling the generator when it't contents will evaluate to






                share|improve this answer















                You are not using a generator correctly if this is the primary use of this code. Use a list comprehension instead of a generator comprehension. Just replace the parentheses with brackets. It evaluates to a list if you don't know.



                array = [1, 2, 2, 4, 5]
                f = [x for x in array if array.count(x) == 2]
                array = [5, 6, 1, 2, 9]

                print(f)
                #[2, 2]


                You are getting this response because of the nature of a generator. You're calling the generator when it't contents will evaluate to







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 7 hours ago

























                answered 7 hours ago









                JabaJaba

                6,958175394




                6,958175394













                • Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                  – Suraj Kothari
                  7 hours ago











                • With your change, list(f) becomes redundant.

                  – Mark Ransom
                  7 hours ago











                • Lol @Mark Ransom, copy paste got me, I edited.

                  – Jaba
                  7 hours ago













                • @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                  – Jaba
                  7 hours ago



















                • Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                  – Suraj Kothari
                  7 hours ago











                • With your change, list(f) becomes redundant.

                  – Mark Ransom
                  7 hours ago











                • Lol @Mark Ransom, copy paste got me, I edited.

                  – Jaba
                  7 hours ago













                • @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                  – Jaba
                  7 hours ago

















                Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                – Suraj Kothari
                7 hours ago





                Thank you. I seem to have used the wrong brackets. But in general using a generator comprehension seems odd.

                – Suraj Kothari
                7 hours ago













                With your change, list(f) becomes redundant.

                – Mark Ransom
                7 hours ago





                With your change, list(f) becomes redundant.

                – Mark Ransom
                7 hours ago













                Lol @Mark Ransom, copy paste got me, I edited.

                – Jaba
                7 hours ago







                Lol @Mark Ransom, copy paste got me, I edited.

                – Jaba
                7 hours ago















                @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                – Jaba
                7 hours ago





                @SurajKothari It is not odd, it's a great tool! It just takes some time to wrap the ole brain around. Do some research you'll find that generators are amazing!

                – Jaba
                7 hours ago











                0














                Generator evaluation is "lazy" -- it doesn't get executed until you actualize it with a proper reference. With your line:



                Look again at your output with the type of f: that object is a generator, not a sequence. It's waiting to be used, an iterator of sorts.



                Your generator isn't evaluated until you start requiring values from it. At that point, it uses the available values at that point, not the point at which it was defined.





                Code to "make it work"



                That depends on what you mean by "make it work". If you want f to be a filtered list, then use a list, not a generator:



                f = [x for x in array if array.count(x) == 2] # Filters original





                share|improve this answer


























                • I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                  – Suraj Kothari
                  7 hours ago
















                0














                Generator evaluation is "lazy" -- it doesn't get executed until you actualize it with a proper reference. With your line:



                Look again at your output with the type of f: that object is a generator, not a sequence. It's waiting to be used, an iterator of sorts.



                Your generator isn't evaluated until you start requiring values from it. At that point, it uses the available values at that point, not the point at which it was defined.





                Code to "make it work"



                That depends on what you mean by "make it work". If you want f to be a filtered list, then use a list, not a generator:



                f = [x for x in array if array.count(x) == 2] # Filters original





                share|improve this answer


























                • I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                  – Suraj Kothari
                  7 hours ago














                0












                0








                0







                Generator evaluation is "lazy" -- it doesn't get executed until you actualize it with a proper reference. With your line:



                Look again at your output with the type of f: that object is a generator, not a sequence. It's waiting to be used, an iterator of sorts.



                Your generator isn't evaluated until you start requiring values from it. At that point, it uses the available values at that point, not the point at which it was defined.





                Code to "make it work"



                That depends on what you mean by "make it work". If you want f to be a filtered list, then use a list, not a generator:



                f = [x for x in array if array.count(x) == 2] # Filters original





                share|improve this answer















                Generator evaluation is "lazy" -- it doesn't get executed until you actualize it with a proper reference. With your line:



                Look again at your output with the type of f: that object is a generator, not a sequence. It's waiting to be used, an iterator of sorts.



                Your generator isn't evaluated until you start requiring values from it. At that point, it uses the available values at that point, not the point at which it was defined.





                Code to "make it work"



                That depends on what you mean by "make it work". If you want f to be a filtered list, then use a list, not a generator:



                f = [x for x in array if array.count(x) == 2] # Filters original






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 7 hours ago

























                answered 7 hours ago









                PrunePrune

                43.2k143456




                43.2k143456













                • I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                  – Suraj Kothari
                  7 hours ago



















                • I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                  – Suraj Kothari
                  7 hours ago

















                I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                – Suraj Kothari
                7 hours ago





                I somewhat understand. Could you show some code to make it work, because I need to re-assign the same list again in the main code.

                – Suraj Kothari
                7 hours ago











                0














                Generators are lazy and your newly defined array is used when you exhaust your generator after redefining. Therefore, the output is correct. A quick fix is to use a list comprehension by replacing parentheses () by brackets .



                Moving on to how better to write your logic, counting a value in a loop has quadratic complexity. For an algorithm that works in linear time, you can use collections.Counter to count values, and keep a copy of your original list:



                from collections import Counter

                array = [1, 2, 2, 4, 5] # original array
                counts = Counter(array) # count each value in array
                old_array = array.copy() # make copy
                array = [5, 6, 1, 2, 9] # updates array

                # order relevant
                res = [x for x in old_array if counts[x] >= 2]
                print(res)
                # [2, 2]

                # order irrelevant
                from itertools import chain
                res = list(chain.from_iterable([x]*count for x, count in counts.items() if count >= 2))
                print(res)
                # [2, 2]


                Notice the second version doesn't even require old_array and is useful if there is no need to maintain ordering of values in your original array.






                share|improve this answer




























                  0














                  Generators are lazy and your newly defined array is used when you exhaust your generator after redefining. Therefore, the output is correct. A quick fix is to use a list comprehension by replacing parentheses () by brackets .



                  Moving on to how better to write your logic, counting a value in a loop has quadratic complexity. For an algorithm that works in linear time, you can use collections.Counter to count values, and keep a copy of your original list:



                  from collections import Counter

                  array = [1, 2, 2, 4, 5] # original array
                  counts = Counter(array) # count each value in array
                  old_array = array.copy() # make copy
                  array = [5, 6, 1, 2, 9] # updates array

                  # order relevant
                  res = [x for x in old_array if counts[x] >= 2]
                  print(res)
                  # [2, 2]

                  # order irrelevant
                  from itertools import chain
                  res = list(chain.from_iterable([x]*count for x, count in counts.items() if count >= 2))
                  print(res)
                  # [2, 2]


                  Notice the second version doesn't even require old_array and is useful if there is no need to maintain ordering of values in your original array.






                  share|improve this answer


























                    0












                    0








                    0







                    Generators are lazy and your newly defined array is used when you exhaust your generator after redefining. Therefore, the output is correct. A quick fix is to use a list comprehension by replacing parentheses () by brackets .



                    Moving on to how better to write your logic, counting a value in a loop has quadratic complexity. For an algorithm that works in linear time, you can use collections.Counter to count values, and keep a copy of your original list:



                    from collections import Counter

                    array = [1, 2, 2, 4, 5] # original array
                    counts = Counter(array) # count each value in array
                    old_array = array.copy() # make copy
                    array = [5, 6, 1, 2, 9] # updates array

                    # order relevant
                    res = [x for x in old_array if counts[x] >= 2]
                    print(res)
                    # [2, 2]

                    # order irrelevant
                    from itertools import chain
                    res = list(chain.from_iterable([x]*count for x, count in counts.items() if count >= 2))
                    print(res)
                    # [2, 2]


                    Notice the second version doesn't even require old_array and is useful if there is no need to maintain ordering of values in your original array.






                    share|improve this answer













                    Generators are lazy and your newly defined array is used when you exhaust your generator after redefining. Therefore, the output is correct. A quick fix is to use a list comprehension by replacing parentheses () by brackets .



                    Moving on to how better to write your logic, counting a value in a loop has quadratic complexity. For an algorithm that works in linear time, you can use collections.Counter to count values, and keep a copy of your original list:



                    from collections import Counter

                    array = [1, 2, 2, 4, 5] # original array
                    counts = Counter(array) # count each value in array
                    old_array = array.copy() # make copy
                    array = [5, 6, 1, 2, 9] # updates array

                    # order relevant
                    res = [x for x in old_array if counts[x] >= 2]
                    print(res)
                    # [2, 2]

                    # order irrelevant
                    from itertools import chain
                    res = list(chain.from_iterable([x]*count for x, count in counts.items() if count >= 2))
                    print(res)
                    # [2, 2]


                    Notice the second version doesn't even require old_array and is useful if there is no need to maintain ordering of values in your original array.







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered 5 hours ago









                    jppjpp

                    95.7k2157109




                    95.7k2157109























                        0














                        The root cause of the problem is that generators are lazy; variables are evaluated each time:



                        >>> l = [1, 2, 2, 4, 5, 5, 5]
                        >>> filtered = (x for x in l if l.count(x) == 2)
                        >>> l = [1, 2, 4, 4, 5, 6, 6]
                        >>> list(filtered)
                        [4]


                        It iterates over the original list and evaluates the condition with the current list. In this case, 4 appeared twice in the new list, causing it to appear in the result. It only appears once in the result because it only appeared once in the original list. The 6s appear twice in the new list, but never appear in the old list and are hence never shown.



                        Full function introspection for the curious (the line with the comment is the important line):



                        >>> l = [1, 2, 2, 4, 5]
                        >>> filtered = (x for x in l if l.count(x) == 2)
                        >>> l = [1, 2, 4, 4, 5, 6, 6]
                        >>> list(filtered)
                        [4]
                        >>> def f(original, new, count):
                        current = original
                        filtered = (x for x in current if current.count(x) == count)
                        current = new
                        return list(filtered)

                        >>> from dis import dis
                        >>> dis(f)
                        2 0 LOAD_FAST 0 (original)
                        3 STORE_DEREF 1 (current)

                        3 6 LOAD_CLOSURE 0 (count)
                        9 LOAD_CLOSURE 1 (current)
                        12 BUILD_TUPLE 2
                        15 LOAD_CONST 1 (<code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>)
                        18 LOAD_CONST 2 ('f.<locals>.<genexpr>')
                        21 MAKE_CLOSURE 0
                        24 LOAD_DEREF 1 (current)
                        27 GET_ITER
                        28 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                        31 STORE_FAST 3 (filtered)

                        4 34 LOAD_FAST 1 (new)
                        37 STORE_DEREF 1 (current)

                        5 40 LOAD_GLOBAL 0 (list)
                        43 LOAD_FAST 3 (filtered)
                        46 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                        49 RETURN_VALUE
                        >>> f.__code__.co_varnames
                        ('original', 'new', 'count', 'filtered')
                        >>> f.__code__.co_cellvars
                        ('count', 'current')
                        >>> f.__code__.co_consts
                        (None, <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>, 'f.<locals>.<genexpr>')
                        >>> f.__code__.co_consts[1]
                        <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>
                        >>> dis(f.__code__.co_consts[1])
                        3 0 LOAD_FAST 0 (.0)
                        >> 3 FOR_ITER 32 (to 38)
                        6 STORE_FAST 1 (x)
                        9 LOAD_DEREF 1 (current) # This loads the current list every time, as opposed to loading a constant.
                        12 LOAD_ATTR 0 (count)
                        15 LOAD_FAST 1 (x)
                        18 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                        21 LOAD_DEREF 0 (count)
                        24 COMPARE_OP 2 (==)
                        27 POP_JUMP_IF_FALSE 3
                        30 LOAD_FAST 1 (x)
                        33 YIELD_VALUE
                        34 POP_TOP
                        35 JUMP_ABSOLUTE 3
                        >> 38 LOAD_CONST 0 (None)
                        41 RETURN_VALUE
                        >>> f.__code__.co_consts[1].co_consts
                        (None,)


                        To reiterate: The list to be iterated is only loaded once. Any closures in the condition or expression, however, are loaded from the enclosing scope each iteration. They are not stored in a constant.



                        The best solution for your problem would be to create a new variable referencing the original list and use that in your generator expression,.






                        share|improve this answer






























                          0














                          The root cause of the problem is that generators are lazy; variables are evaluated each time:



                          >>> l = [1, 2, 2, 4, 5, 5, 5]
                          >>> filtered = (x for x in l if l.count(x) == 2)
                          >>> l = [1, 2, 4, 4, 5, 6, 6]
                          >>> list(filtered)
                          [4]


                          It iterates over the original list and evaluates the condition with the current list. In this case, 4 appeared twice in the new list, causing it to appear in the result. It only appears once in the result because it only appeared once in the original list. The 6s appear twice in the new list, but never appear in the old list and are hence never shown.



                          Full function introspection for the curious (the line with the comment is the important line):



                          >>> l = [1, 2, 2, 4, 5]
                          >>> filtered = (x for x in l if l.count(x) == 2)
                          >>> l = [1, 2, 4, 4, 5, 6, 6]
                          >>> list(filtered)
                          [4]
                          >>> def f(original, new, count):
                          current = original
                          filtered = (x for x in current if current.count(x) == count)
                          current = new
                          return list(filtered)

                          >>> from dis import dis
                          >>> dis(f)
                          2 0 LOAD_FAST 0 (original)
                          3 STORE_DEREF 1 (current)

                          3 6 LOAD_CLOSURE 0 (count)
                          9 LOAD_CLOSURE 1 (current)
                          12 BUILD_TUPLE 2
                          15 LOAD_CONST 1 (<code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>)
                          18 LOAD_CONST 2 ('f.<locals>.<genexpr>')
                          21 MAKE_CLOSURE 0
                          24 LOAD_DEREF 1 (current)
                          27 GET_ITER
                          28 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                          31 STORE_FAST 3 (filtered)

                          4 34 LOAD_FAST 1 (new)
                          37 STORE_DEREF 1 (current)

                          5 40 LOAD_GLOBAL 0 (list)
                          43 LOAD_FAST 3 (filtered)
                          46 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                          49 RETURN_VALUE
                          >>> f.__code__.co_varnames
                          ('original', 'new', 'count', 'filtered')
                          >>> f.__code__.co_cellvars
                          ('count', 'current')
                          >>> f.__code__.co_consts
                          (None, <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>, 'f.<locals>.<genexpr>')
                          >>> f.__code__.co_consts[1]
                          <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>
                          >>> dis(f.__code__.co_consts[1])
                          3 0 LOAD_FAST 0 (.0)
                          >> 3 FOR_ITER 32 (to 38)
                          6 STORE_FAST 1 (x)
                          9 LOAD_DEREF 1 (current) # This loads the current list every time, as opposed to loading a constant.
                          12 LOAD_ATTR 0 (count)
                          15 LOAD_FAST 1 (x)
                          18 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                          21 LOAD_DEREF 0 (count)
                          24 COMPARE_OP 2 (==)
                          27 POP_JUMP_IF_FALSE 3
                          30 LOAD_FAST 1 (x)
                          33 YIELD_VALUE
                          34 POP_TOP
                          35 JUMP_ABSOLUTE 3
                          >> 38 LOAD_CONST 0 (None)
                          41 RETURN_VALUE
                          >>> f.__code__.co_consts[1].co_consts
                          (None,)


                          To reiterate: The list to be iterated is only loaded once. Any closures in the condition or expression, however, are loaded from the enclosing scope each iteration. They are not stored in a constant.



                          The best solution for your problem would be to create a new variable referencing the original list and use that in your generator expression,.






                          share|improve this answer




























                            0












                            0








                            0







                            The root cause of the problem is that generators are lazy; variables are evaluated each time:



                            >>> l = [1, 2, 2, 4, 5, 5, 5]
                            >>> filtered = (x for x in l if l.count(x) == 2)
                            >>> l = [1, 2, 4, 4, 5, 6, 6]
                            >>> list(filtered)
                            [4]


                            It iterates over the original list and evaluates the condition with the current list. In this case, 4 appeared twice in the new list, causing it to appear in the result. It only appears once in the result because it only appeared once in the original list. The 6s appear twice in the new list, but never appear in the old list and are hence never shown.



                            Full function introspection for the curious (the line with the comment is the important line):



                            >>> l = [1, 2, 2, 4, 5]
                            >>> filtered = (x for x in l if l.count(x) == 2)
                            >>> l = [1, 2, 4, 4, 5, 6, 6]
                            >>> list(filtered)
                            [4]
                            >>> def f(original, new, count):
                            current = original
                            filtered = (x for x in current if current.count(x) == count)
                            current = new
                            return list(filtered)

                            >>> from dis import dis
                            >>> dis(f)
                            2 0 LOAD_FAST 0 (original)
                            3 STORE_DEREF 1 (current)

                            3 6 LOAD_CLOSURE 0 (count)
                            9 LOAD_CLOSURE 1 (current)
                            12 BUILD_TUPLE 2
                            15 LOAD_CONST 1 (<code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>)
                            18 LOAD_CONST 2 ('f.<locals>.<genexpr>')
                            21 MAKE_CLOSURE 0
                            24 LOAD_DEREF 1 (current)
                            27 GET_ITER
                            28 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            31 STORE_FAST 3 (filtered)

                            4 34 LOAD_FAST 1 (new)
                            37 STORE_DEREF 1 (current)

                            5 40 LOAD_GLOBAL 0 (list)
                            43 LOAD_FAST 3 (filtered)
                            46 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            49 RETURN_VALUE
                            >>> f.__code__.co_varnames
                            ('original', 'new', 'count', 'filtered')
                            >>> f.__code__.co_cellvars
                            ('count', 'current')
                            >>> f.__code__.co_consts
                            (None, <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>, 'f.<locals>.<genexpr>')
                            >>> f.__code__.co_consts[1]
                            <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>
                            >>> dis(f.__code__.co_consts[1])
                            3 0 LOAD_FAST 0 (.0)
                            >> 3 FOR_ITER 32 (to 38)
                            6 STORE_FAST 1 (x)
                            9 LOAD_DEREF 1 (current) # This loads the current list every time, as opposed to loading a constant.
                            12 LOAD_ATTR 0 (count)
                            15 LOAD_FAST 1 (x)
                            18 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            21 LOAD_DEREF 0 (count)
                            24 COMPARE_OP 2 (==)
                            27 POP_JUMP_IF_FALSE 3
                            30 LOAD_FAST 1 (x)
                            33 YIELD_VALUE
                            34 POP_TOP
                            35 JUMP_ABSOLUTE 3
                            >> 38 LOAD_CONST 0 (None)
                            41 RETURN_VALUE
                            >>> f.__code__.co_consts[1].co_consts
                            (None,)


                            To reiterate: The list to be iterated is only loaded once. Any closures in the condition or expression, however, are loaded from the enclosing scope each iteration. They are not stored in a constant.



                            The best solution for your problem would be to create a new variable referencing the original list and use that in your generator expression,.






                            share|improve this answer















                            The root cause of the problem is that generators are lazy; variables are evaluated each time:



                            >>> l = [1, 2, 2, 4, 5, 5, 5]
                            >>> filtered = (x for x in l if l.count(x) == 2)
                            >>> l = [1, 2, 4, 4, 5, 6, 6]
                            >>> list(filtered)
                            [4]


                            It iterates over the original list and evaluates the condition with the current list. In this case, 4 appeared twice in the new list, causing it to appear in the result. It only appears once in the result because it only appeared once in the original list. The 6s appear twice in the new list, but never appear in the old list and are hence never shown.



                            Full function introspection for the curious (the line with the comment is the important line):



                            >>> l = [1, 2, 2, 4, 5]
                            >>> filtered = (x for x in l if l.count(x) == 2)
                            >>> l = [1, 2, 4, 4, 5, 6, 6]
                            >>> list(filtered)
                            [4]
                            >>> def f(original, new, count):
                            current = original
                            filtered = (x for x in current if current.count(x) == count)
                            current = new
                            return list(filtered)

                            >>> from dis import dis
                            >>> dis(f)
                            2 0 LOAD_FAST 0 (original)
                            3 STORE_DEREF 1 (current)

                            3 6 LOAD_CLOSURE 0 (count)
                            9 LOAD_CLOSURE 1 (current)
                            12 BUILD_TUPLE 2
                            15 LOAD_CONST 1 (<code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>)
                            18 LOAD_CONST 2 ('f.<locals>.<genexpr>')
                            21 MAKE_CLOSURE 0
                            24 LOAD_DEREF 1 (current)
                            27 GET_ITER
                            28 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            31 STORE_FAST 3 (filtered)

                            4 34 LOAD_FAST 1 (new)
                            37 STORE_DEREF 1 (current)

                            5 40 LOAD_GLOBAL 0 (list)
                            43 LOAD_FAST 3 (filtered)
                            46 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            49 RETURN_VALUE
                            >>> f.__code__.co_varnames
                            ('original', 'new', 'count', 'filtered')
                            >>> f.__code__.co_cellvars
                            ('count', 'current')
                            >>> f.__code__.co_consts
                            (None, <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>, 'f.<locals>.<genexpr>')
                            >>> f.__code__.co_consts[1]
                            <code object <genexpr> at 0x02DD36B0, file "<pyshell#17>", line 3>
                            >>> dis(f.__code__.co_consts[1])
                            3 0 LOAD_FAST 0 (.0)
                            >> 3 FOR_ITER 32 (to 38)
                            6 STORE_FAST 1 (x)
                            9 LOAD_DEREF 1 (current) # This loads the current list every time, as opposed to loading a constant.
                            12 LOAD_ATTR 0 (count)
                            15 LOAD_FAST 1 (x)
                            18 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
                            21 LOAD_DEREF 0 (count)
                            24 COMPARE_OP 2 (==)
                            27 POP_JUMP_IF_FALSE 3
                            30 LOAD_FAST 1 (x)
                            33 YIELD_VALUE
                            34 POP_TOP
                            35 JUMP_ABSOLUTE 3
                            >> 38 LOAD_CONST 0 (None)
                            41 RETURN_VALUE
                            >>> f.__code__.co_consts[1].co_consts
                            (None,)


                            To reiterate: The list to be iterated is only loaded once. Any closures in the condition or expression, however, are loaded from the enclosing scope each iteration. They are not stored in a constant.



                            The best solution for your problem would be to create a new variable referencing the original list and use that in your generator expression,.







                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited 3 hours ago

























                            answered 3 hours ago









                            Solomon UckoSolomon Ucko

                            6271719




                            6271719






























                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Stack Overflow!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54245618%2funexpected-behaviour-with-python-generator%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Understanding the information contained in the Deep Space Network XML data?

                                Ross-on-Wye

                                Eastern Orthodox Church