Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Docs: add note about dataclass items and loaders

Eugenio Lacuesta3efea98e0518
diff --git a/docs/topics/loaders.rst b/docs/topics/loaders.rst
index 6645bf123..d70e03ad4 100644
--- a/docs/topics/loaders.rst
+++ b/docs/topics/loaders.rst
@@ -26,7 +26,7 @@ Using Item Loaders to populate items
To use an Item Loader, you must first instantiate it. You can either
instantiate it with an :ref:`item object <topics-items>` or without one, in which
case an instance of :class:`~scrapy.item.Item` is automatically created in the
-Item Loader ``__init__`` method using the :class:`~scrapy.item.Item` subclass
+Item Loader ``__init__`` method using the :ref:`item <topics-items>` class
specified in the :attr:`ItemLoader.default_item_class` attribute.
Then, you start collecting values into the Item Loader, typically using
@@ -76,6 +76,41 @@ called which actually returns the item populated with the data
previously extracted and collected with the :meth:`~ItemLoader.add_xpath`,
:meth:`~ItemLoader.add_css`, and :meth:`~ItemLoader.add_value` calls.
+
+.. _topics-loaders-dataclass:
+
+Working with dataclass items
+============================
+
+By default, :ref:`dataclass items <dataclass-items>` require all fields to be
+passed when created. This could be an issue when using dataclass items with
+item loaders, since fields could be populated incrementally.
+
+Given the way that item loaders store data internally, the recommended approach
+to overcome this is to define items using the :func:`~dataclasses.field`
+function, with ``list`` as the ``default_factory`` argument::
+
+ from dataclasses import dataclass, field
+
+ @dataclass
+ class InventoryItem:
+ name: str = field(default_factory=list)
+ price: float = field(default_factory=list)
+ stock: int = field(default_factory=list)
+
+Note that in order to keep the example simple, the types do not match
+completely. A more accurate but verbose definition would be::
+
+ from dataclasses import dataclass, field
+ from typing import List, Union
+
+ @dataclass
+ class InventoryItem:
+ name: Union[str, List[str]] = field(default_factory=list)
+ price: Union[float, List[float]] = field(default_factory=list)
+ stock: Union[int, List[int]] = field(default_factory=list)
+
+
.. _topics-loaders-processors:
Input and Output processors