Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 77 additions & 26 deletions Doc/reference/expressions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -956,39 +956,90 @@ Generator expressions
pair: object; generator
single: () (parentheses); generator expression

A generator expression is a compact generator notation in parentheses:
The syntax for :dfn:`generator expressions` is the same as for
list :ref:`comprehensions <comprehensions>`, except that they are enclosed in
parentheses instead of brackets.
For example::

.. productionlist:: python-grammar
generator_expression: "(" `comprehension` ")"
>>> iterator = (x ** 2 for x in range(10))
>>> iterator
<generator object <genexpr> at ...>

At runtime, a generator expression evaluates to a :term:`generator iterator`
which yields the same values as the corresponding list comprehension::

>>> list(iterator)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Thus, the example above is roughly equivalent to defining and calling
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make it less rough, we could pass an iterator?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean; where should we pass it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant something like:

def make_generator_of_squares(iterator):
   for x in iterator:
       yield x ** 2

make_generator_of_squares(iter(range(10)))

That way, it's also evaluated immediately like in generator expressions.

the following generator function::

def make_generator_of_squares(iterable):
for x in iterable:
yield x ** 2

make_generator_of_squares(range(10))

The enclosing parentheses can be omitted in calls with only one
positional argument.
Comment on lines +983 to +984
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The enclosing parentheses can be omitted in calls with only one
positional argument.
The enclosing parentheses can be omitted in calls when it is
the only argument.

Proof by counterexample: sum(x ** 2 for x in range(10), start=3) has only one pos-arg.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The single argument must be positional. For example, sum(iterable=x ** 2 for x in range(10)) is invalid syntax.

Would this work?

Suggested change
The enclosing parentheses can be omitted in calls with only one
positional argument.
The enclosing parentheses can be omitted in calls when the generator
expression is the only positional argument and there are no keyword
arguments.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine.

See the :ref:`Calls section <calls>` for details.
For example::

# The parentheses after `sum` are part of the call syntax:
>>> sum(x ** 2 for x in range(10))
285

# The generator needs its own parentheses if it's not the only argument:
>>> sum((x ** 2 for x in range(10)), start=1000)
1285

A generator expression yields a new generator object. Its syntax is the same as
for comprehensions, except that it is enclosed in parentheses instead of
brackets or curly braces.

Variables used in the generator expression are evaluated lazily when the
:meth:`~generator.__next__` method is called for the generator object (in the same
fashion as normal generators). However, the iterable expression in the
leftmost :keyword:`!for` clause is immediately evaluated, and the
:term:`iterator` is immediately created for that iterable, so that an error
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's moved around and reworded, see "The iterable expression in the leftmost :keyword:!for clause is evaluated immediately" in the new text.
For "Subsequent for clauses" the new wording is "All other expressions" -- it's for, if but also the "result" expression.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my unclear comment, I was referring specifically to keeping the sentence on L972: "iterator is immediately created for that iterable"

(Which in turn made me think of the comment above :-)

produced while creating the iterator will be emitted at the point where the generator expression
is defined, rather than at the point where the first value is retrieved.
Subsequent :keyword:`!for` clauses and any filter condition in the leftmost
:keyword:`!for` clause cannot be evaluated in the enclosing scope as they may
depend on the values obtained from the leftmost iterable. For example:
``(x*y for x in range(10) for y in range(x, x+10))``.

The parentheses can be omitted on calls with only one argument. See section
:ref:`calls` for details.
The iterable expression in the leftmost :keyword:`!for` clause is
evaluated immediately, so that an error raised by this expression will be
emitted at the point where the generator expression is defined,
rather than at the point where the first value is retrieved::

>>> (x ** 2 for x in nonexistent_iterable)
Traceback (most recent call last):
...
NameError: name 'nonexistent_iterable' is not defined

All other expressions are evaluated lazily, in the same fashion as normal
generators (that is, when the iterator is asked to yield a value)::

>>> iterator = (nonexistent_value for x in range(10))
>>> iterator
<generator object <genexpr> at ...>
>>> list(iterator)
Traceback (most recent call last):
...
NameError: name 'nonexistent_value' is not defined

::

>>> iterator = (x * y for x in range(10) for y in nonexistent_iterable)
>>> iterator
<generator object <genexpr> at ...>
>>> list(iterator)
Traceback (most recent call last):
...
NameError: name 'nonexistent_iterable' is not defined

To avoid interfering with the expected operation of the generator expression
itself, ``yield`` and ``yield from`` expressions are prohibited in the
implicitly defined generator.
itself, ``yield`` and ``yield from`` expressions are prohibited inside
the implicitly nested scope.

If a generator expression contains either :keyword:`!async for`
clauses or :keyword:`await` expressions it is called an
:dfn:`asynchronous generator expression`. An asynchronous generator
expression returns a new asynchronous generator object,
which is an asynchronous iterator (see :ref:`async-iterators`).
:dfn:`asynchronous generator expression`.
An asynchronous generator expression returns a new asynchronous generator
object, which is an asynchronous iterator (see :ref:`async-iterators`).

The formal grammar for generator expressions is:

.. grammar-snippet::
:group: python-grammar

generator_expression: "(" `comprehension` ")"

.. versionadded:: 3.6
Asynchronous generator expressions were introduced.
Expand Down
Loading