[FEA] Multi-node Out of Core Streaming KMeans API#2066
Conversation
…nto combine-batch
…nto combine-batch
…nto combine-batch
|
/ok to tesst ccd5e43 |
|
/ok to test ccd5e43 |
…into mnmg-streaming
|
/ok to test 21864d2 |
|
/ok to test 4da8077 |
|
/ok to test a0edd32 |
|
/ok to test c8db7ff |
|
/ok to test fe98618 |
| if(Python_FOUND) | ||
| include(${rapids-cmake-dir}/cython-core/find_prefix_paths.cmake) | ||
| rapids_cython_find_prefix_paths("${Python_EXECUTABLE}" _cuvs_wheel_prefix_paths) | ||
| set_property(GLOBAL PROPERTY FIND_LIBRARY_USE_LIB64_PATHS TRUE) |
There was a problem hiding this comment.
I don’t fully understand what this code path is aiming to do, but conda doesn’t use lib64 paths iirc. Normally I would assume this is used in wheels, but we don’t build C++ tests for wheels. What’s going on here?
There was a problem hiding this comment.
I had done that to ensure that if those dependencies are coming from pip, they are found. Because find_package was failing if I get pylibraft from pip. I agree that we no longer do this in our conda builds, so it is not needed
| NAME CLUSTER_KMEANS_MG_TEST | ||
| PATH cluster/kmeans_mg.cu |
There was a problem hiding this comment.
Please mark all multi-GPU tests as SERIAL. You probably don’t want to run two 2-GPU tests concurrently on a 4-GPU system. It just gets messy. We recently discussed this topic for cudf and that was the right choice. rapidsai/cudf#23188
…into mnmg-streaming
|
/ok to test 07b5a39 |
There was a problem hiding this comment.
The SNMG batched KMeans C/Python API was merged before and unfortunately now calls the single-GPU path after this PR. We should fix this.
|
/ok to test 78a4278 |
|
/merge |
8 participants
Merge after #2015 and #2017
Allows a stream of input matrices per worker, that are further batched using the
streaming_batch_sizeparameter. Reasoning: We should be able to supply dask partitions (on host) directly without having to concatenate them into one consolidated matrix.As a part of this PR, we also unify the multi-GPU implementations into one (earlier the out of core implementation was separate).
Tests: We get rid of the separate out of core test file. The single MG testing unit is taking care of both out of core and on device matrices.