Can not pickle `torch.utils.data.DataLoader`

[k82cn]

Hi team,

I'm working on some distributed system for AI, and I'd like to pickle an object to transfer it. But it seems I can not pickle DataLoader, is there any suggestions or best practice for that?

  File "/opt/examples/ps/.venv/lib/python3.12/site-packages/flamepy/client.py", line 325, in create_session
    object_ref = put_object(session_id, attrs.common_data)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/examples/ps/.venv/lib/python3.12/site-packages/flamepy/cache.py", line 74, in put_object
    data = cloudpickle.dumps(obj, protocol=cloudpickle.DEFAULT_PROTOCOL)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/examples/ps/.venv/lib/python3.12/site-packages/cloudpickle/cloudpickle.py", line 1544, in dumps
    cp.dump(obj)
  File "/opt/examples/ps/.venv/lib/python3.12/site-packages/cloudpickle/cloudpickle.py", line 1313, in dump
    return super().dump(obj)
           ^^^^^^^^^^^^^^^^^
  File "/opt/examples/ps/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 761, in __getstate__
    raise NotImplementedError("{} cannot be pickled", self.__class__.__name__)
NotImplementedError: ('{} cannot be pickled', '_SingleProcessDataLoaderIter')