Problem
With on_csv_write="update", the second write for a given run_id crashes when any column is empty in every existing CSV row. This is easy to hit in practice: an OfflineEmissionsTracker leaves longitude/latitude empty, and gpu_count/gpu_model are empty on CPU-only machines. Any run that writes twice — a flush() followed by stop(), or a tracker with measure_power_secs scheduling — crashes on the second write.
Reproduction
import tempfile
from codecarbon.output_methods.emissions_data import EmissionsData
from codecarbon.output_methods.file import FileOutput
d = EmissionsData(
timestamp="2023-01-01T00:00:00", project_name="p", run_id="r", experiment_id="e",
duration=10, emissions=0.5, emissions_rate=0.05, cpu_power=20, gpu_power=0,
ram_power=5, cpu_energy=200, gpu_energy=0, ram_energy=50, energy_consumed=250,
water_consumed=0.1, country_name="Testland", country_iso_code="TS", region="R",
cloud_provider="", cloud_region="", os="TestOS", python_version="3.8",
codecarbon_version="2.0", cpu_count=4, cpu_model="CPU", gpu_count=None,
gpu_model=None, longitude="", latitude="", ram_total_size=16, tracking_mode="machine",
)
f = FileOutput("test.csv", tempfile.mkdtemp(), on_csv_write="update")
f.out(d, None)
f.out(d, None) # raises
Root cause
codecarbon/output_methods/file.py:122, in the branch handling exactly one existing row with the current run_id:
update_values = {}
for col, val in dict(total.values).items():
update_values[col] = df[col].dtype.type(val)
df[col].dtype.type(val) coerces the incoming value to whatever dtype pandas inferred when reading the existing CSV back. A column that is empty in every existing row is read back as float64, so this evaluates numpy.float64("") or numpy.float64(None).
Expected vs actual
Expected: the existing row for the run is replaced by the new values, empty columns included.
Actual:
File "codecarbon/output_methods/file.py", line 122, in out
update_values[col] = df[col].dtype.type(val)
ValueError: could not convert string to float: ''
(None instead of "" gives TypeError: float() argument must be a string or a real number, not 'NoneType'.)
The coercion serves no purpose here — pandas can assign the values directly, or the row can simply be rebuilt.
Problem
With
on_csv_write="update", the second write for a givenrun_idcrashes when any column is empty in every existing CSV row. This is easy to hit in practice: anOfflineEmissionsTrackerleaveslongitude/latitudeempty, andgpu_count/gpu_modelare empty on CPU-only machines. Any run that writes twice — aflush()followed bystop(), or a tracker withmeasure_power_secsscheduling — crashes on the second write.Reproduction
Root cause
codecarbon/output_methods/file.py:122, in the branch handling exactly one existing row with the currentrun_id:df[col].dtype.type(val)coerces the incoming value to whatever dtype pandas inferred when reading the existing CSV back. A column that is empty in every existing row is read back asfloat64, so this evaluatesnumpy.float64("")ornumpy.float64(None).Expected vs actual
Expected: the existing row for the run is replaced by the new values, empty columns included.
Actual:
(
Noneinstead of""givesTypeError: float() argument must be a string or a real number, not 'NoneType'.)The coercion serves no purpose here — pandas can assign the values directly, or the row can simply be rebuilt.