Conversation
| updateconf_command.push_back("enable_user_defined_functions:true"); | ||
| } | ||
|
|
||
| // Create Cassandra version specific updated (C* 3.0+) | ||
| if (cassandra_version >= "3.0.0") { | ||
| updateconf_command.push_back("enable_scripted_user_defined_functions:true"); |
There was a problem hiding this comment.
This doesn't seem to be in option in Cassandra 5.0+ but as far as I can tell we don't actually used scripted UDFs anyways so we can just delete it
There was a problem hiding this comment.
It's still defined, it's just been renamed because of the conversion that happened with C* 4.1 (the reason we have to do all this translation. The reference is here.
You should be able to keep this in and just let the for loop calling translate_config_for_version() below take care of this.
There was a problem hiding this comment.
If you look at https://apache.googlesource.com/cassandra/+/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java, you'll see enable_scripted_user_defined_functions was actually removed in 5.0+. But like I said we don't use JS UDFs anyways so we don't need it in the first place (of course if I'm wrong do lmk)
|
|
||
| for (const auto& i : SUFFIXES) { | ||
| std::string suffix = i[0]; | ||
| if (key.size() >= suffix.size() && |
There was a problem hiding this comment.
I noticed there was are starts_with and ends_with functions but then I'd have to use datastax::StringRef instead of just std::string.
Which I still could if you think it's cleaner to be honest.
| * @return The pair translated if renamed and cassandra_version >= 4.1.0; | ||
| * otherwise key_value unchanged | ||
| */ | ||
| std::string translate_config_for_version(const std::string& key_value, |
There was a problem hiding this comment.
Ideally this would just be a static free-floating function in the cpp file itself but
- the other helper functions were defined in the header as part of the class
- the other helper functions were static had TODO comments to remove the static modifiers
so I just kept it like this for consistency 🤷
|
|
||
| // Verify virtual table's metadata | ||
| EXPECT_EQ(cass_table_meta_column_count(table_meta.get()), 8u); | ||
| EXPECT_GE(cass_table_meta_column_count(table_meta.get()), is_4_1_or_later ? 9u : 8u); |
There was a problem hiding this comment.
I think it was 10 columns for 5.x iirc? Either way I didn't find it very productive to essentially just map every cassandra version to the number of columns for a table for just a single test 🤷
There was a problem hiding this comment.
If we have three different answers (one for 5.0.x, one for 4.1.x and one for anything earlier) it might make sense to bump this to a helper function which returns the expected value... I think that was done for one of the other drivers (although I can't find it right now).
If it's just a choice between 8 or 9 then yeah, I don't think a helper function is really necessary here; this code seems pretty clear as it stands.
There was a problem hiding this comment.
I thought of this but wasn't sure if it was worth adding conditional test cases for each version because that could arguably be a slippery slope in terms of maintainability.
If you think it's worth it for the sake of completeness then I could still do so, your call here
| '3.0', // Previous Apache Cassandra | ||
| '3.11', // Current Apache Cassandra | ||
| '4.0', // Development Apache Cassandra | ||
| '3.11', // Previous Apache Cassandra |
There was a problem hiding this comment.
Not sure which exact versions we want to test so I just left them all here for now...
There was a problem hiding this comment.
We normally aim to support platforms (operating systems, JDK versions, Python runtimes etc.) that aren't EOL at time of release. That policy doesn't apply directly to the underlying Cassandra version but in practice we basically do the same. So Java currently supports C* 4.0 and up (since everything older is EOL) and my plan was to do the same here.
DSE certainly complicates that story. I haven't done a look recently to see where we stand on supported DSE versions but I think DSE 5.1, 6.8 and prolly 6.9 is likely adequate. Would be nice if we could get HCD 1.0 in there as well... did you do any testing with that?
absurdfarce
left a comment
There was a problem hiding this comment.
Nice work @toptobes! A few minor things in the review, nothing that's really a blocker.
Have you tested this much with DSE?
| '3.0', // Previous Apache Cassandra | ||
| '3.11', // Current Apache Cassandra | ||
| '4.0', // Development Apache Cassandra | ||
| '3.11', // Previous Apache Cassandra |
There was a problem hiding this comment.
We normally aim to support platforms (operating systems, JDK versions, Python runtimes etc.) that aren't EOL at time of release. That policy doesn't apply directly to the underlying Cassandra version but in practice we basically do the same. So Java currently supports C* 4.0 and up (since everything older is EOL) and my plan was to do the same here.
DSE certainly complicates that story. I haven't done a look recently to see where we stand on supported DSE versions but I think DSE 5.1, 6.8 and prolly 6.9 is likely adequate. Would be nice if we could get HCD 1.0 in there as well... did you do any testing with that?
| updateconf_command.push_back("enable_user_defined_functions:true"); | ||
| } | ||
|
|
||
| // Create Cassandra version specific updated (C* 3.0+) | ||
| if (cassandra_version >= "3.0.0") { | ||
| updateconf_command.push_back("enable_scripted_user_defined_functions:true"); |
There was a problem hiding this comment.
It's still defined, it's just been renamed because of the conversion that happened with C* 4.1 (the reason we have to do all this translation. The reference is here.
You should be able to keep this in and just let the for loop calling translate_config_for_version() below take care of this.
| if (cassandra_version >= "4.0.0" && !is_dse()) { | ||
| updateconf_command.push_back("enable_materialized_views:true"); | ||
| updateconf_command.push_back("enable_user_defined_functions:true"); | ||
| } |
There was a problem hiding this comment.
Boy, there are a lot of hard-coded cassandra.yaml configs that get updated here for very obsolete C* versions. I see fixed config settings for pre-2.0 C*... yikes. Those can all be deleted, or at least the ones for anything before C* 4.0 can be (assuming we're using OSS C* as our minimum here as discussed in another comment).
Specifically I'm thinking all of these can be removed and these can just be made part of the default.
There was a problem hiding this comment.
lol alright will do
There was a problem hiding this comment.
Will put this on hold until we decide exactly which versions we want to keep testing against
| for (size_t i = 0; i < updateconf_command.size(); ++i) { | ||
| updateconf_command[i] = translate_config_for_version(updateconf_command[i], cassandra_version); | ||
| } | ||
|
|
There was a problem hiding this comment.
Semi-nit: this forces us to loop through the array twice, once when creating it and again to update everything. Maybe just create a helper function which calls push_back(translate_config_for_version()) and then use that helper to add each setting here? That way you can translate as you go.
There was a problem hiding this comment.
I did that at first but because there's no nested functions and I didn't see lambdas used much in many other areas of the codebase (i.e. so no clean closures or currying) I just stuck with this way because I didn't like having to pass all the context into the translation function at every call.
It's not performance sensitive nor is it a lot of elements so it's fine in this context. If we wanted pure performance there's plenty more we could do here beyond solving O(2N) 🙂
|
|
||
| // Verify virtual table's metadata | ||
| EXPECT_EQ(cass_table_meta_column_count(table_meta.get()), 8u); | ||
| EXPECT_GE(cass_table_meta_column_count(table_meta.get()), is_4_1_or_later ? 9u : 8u); |
There was a problem hiding this comment.
If we have three different answers (one for 5.0.x, one for 4.1.x and one for anything earlier) it might make sense to bump this to a helper function which returns the expected value... I think that was done for one of the other drivers (although I can't find it right now).
If it's just a choice between 8 or 9 then yeah, I don't think a helper function is really necessary here; this code seems pretty clear as it stands.
No description provided.