[2024-11-13 16:26:30,199] : DEBUG : abi_l1b_geotiff : _prepare_initial_logging : Starting script with arguments: -r abi_l1b -w geotiff -p true_color_night -f /input/OR_ABI-L1b-RadC-M6C02_G18_s20243181616173_e20243181618546_c20243181618580.nc /input/OR_ABI-L1b-RadC-M6C02_G18_s20243181621173_e20243181623546_c20243181623574.nc /input/OR_ABI-L1b-RadC-M6C03_G18_s20243181616173_e20243181618546_c20243181618583.nc /input/OR_ABI-L1b-RadC-M6C03_G18_s20243181621173_e20243181623546_c20243181623598.nc -vv [2024-11-13 16:26:30,200] : INFO : abi_l1b_geotiff : _run_processing : Sorting and reading input files... [2024-11-13 16:26:30,200] : DEBUG : abi_l1b_geotiff : _set_preferred_chunk_size : Setting preferred chunk size to 1356 pixels or 14MiB [2024-11-13 16:26:30,210] : DEBUG : satpy.readers.yaml_reader : load_yaml_configs : Reading ('/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/etc/readers/abi_l1b.yaml',) [2024-11-13 16:26:30,287] : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to abi_l1b: ['/input/OR_ABI-L1b-RadC-M6C02_G18_s20243181621173_e20243181623546_c20243181623574.nc', '/input/OR_ABI-L1b-RadC-M6C02_G18_s20243181616173_e20243181618546_c20243181618580.nc', '/input/OR_ABI-L1b-RadC-M6C03_G18_s20243181616173_e20243181618546_c20243181618583.nc', '/input/OR_ABI-L1b-RadC-M6C03_G18_s20243181621173_e20243181623546_c20243181623598.nc'] [2024-11-13 16:26:30,513] : PID 150 : DEBUG : polar2grid.core.script_utils : rename_log_file : Log renamed from '/output/abi_l1b_geotiff_fail.log' to 'abi_l1b_geotiff_20241113_161617.log' [2024-11-13 16:26:30,514] : PID 150 : INFO : abi_l1b_geotiff : _run_processing : Loading product metadata from files... [2024-11-13 16:26:30,515] : PID 150 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file abi.yaml [2024-11-13 16:26:30,906] : PID 150 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file visir.yaml [2024-11-13 16:26:30,990] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 16:26:30,998] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_azimuth_angle', calibration='reflectance') [2024-11-13 16:26:30,999] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_zenith_angle', calibration='reflectance') [2024-11-13 16:26:30,999] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_azimuth_angle', calibration='reflectance') [2024-11-13 16:26:31,000] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 16:26:31,004] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 16:26:31,012] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_azimuth_angle', calibration='reflectance') [2024-11-13 16:26:31,012] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_zenith_angle', calibration='reflectance') [2024-11-13 16:26:31,013] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_azimuth_angle', calibration='reflectance') [2024-11-13 16:26:31,013] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 16:26:31,017] : PID 150 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 16:26:31,019] : PID 150 : WARNING : satpy.readers.yaml_reader : _get_file_handlers : Required file type 'c14' not found or loaded for 'C14' [2024-11-13 16:26:31,019] : PID 150 : WARNING : satpy.readers.yaml_reader : _get_file_handlers : Required file type 'c01' not found or loaded for 'C01' [2024-11-13 16:26:31,020] : PID 150 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C03. [2024-11-13 16:26:31,085] : PID 150 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C03. [2024-11-13 16:26:31,162] : PID 150 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C02. [2024-11-13 16:26:31,216] : PID 150 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C02. [2024-11-13 16:26:31,283] : PID 150 : DEBUG : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DataID(name='C01', wavelength=WavelengthRange(min=0.45, central=0.47, max=0.49, unit='µm'), resolution=1000, calibration=, modifiers=('sunz_corrected',))': 'DataID(name='C01', wavelength=WavelengthRange(min=0.45, central=0.47, max=0.49, unit='µm'), resolution=1000, calibration=, modifiers=())' [2024-11-13 16:26:31,284] : PID 150 : DEBUG : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DataID(name='C01', wavelength=WavelengthRange(min=0.45, central=0.47, max=0.49, unit='µm'), resolution=1000, calibration=, modifiers=('sunz_corrected', 'rayleigh_corrected'))': 'DataID(name='C01', wavelength=WavelengthRange(min=0.45, central=0.47, max=0.49, unit='µm'), resolution=1000, calibration=, modifiers=('sunz_corrected',))' [2024-11-13 16:26:31,284] : PID 150 : DEBUG : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DataID(name='green')': 'DataID(name='C01', wavelength=WavelengthRange(min=0.45, central=0.47, max=0.49, unit='µm'), resolution=1000, calibration=, modifiers=('sunz_corrected', 'rayleigh_corrected'))' [2024-11-13 16:26:31,286] : PID 150 : DEBUG : satpy.modifiers.geometry : __call__ : Applying sun zen correction [2024-11-13 16:26:31,286] : PID 150 : DEBUG : satpy.modifiers.geometry : __call__ : Computing sun zenith angles. [2024-11-13 16:26:31,301] : PID 150 : DEBUG : satpy.modifiers.geometry : _apply_correction : Apply the standard sun-zenith correction [1/cos(sunz)] [2024-11-13 16:26:31,316] : PID 150 : DEBUG : h5py._conv : : Creating converter from 7 to 5 [2024-11-13 16:26:31,317] : PID 150 : DEBUG : h5py._conv : : Creating converter from 5 to 7 [2024-11-13 16:26:31,317] : PID 150 : DEBUG : h5py._conv : : Creating converter from 7 to 5 [2024-11-13 16:26:31,317] : PID 150 : DEBUG : h5py._conv : : Creating converter from 5 to 7 [2024-11-13 16:26:31,370] : PID 150 : INFO : satpy.modifiers.atmosphere : __call__ : Removing Rayleigh scattering with atmosphere 'us-standard' and aerosol type 'rayleigh_only' for 'C02' [2024-11-13 16:26:31,372] : PID 150 : INFO : pyspectral.rayleigh : __init__ : Atmosphere chosen: us-standard [2024-11-13 16:26:31,375] : PID 150 : DEBUG : pyspectral.rayleigh : __init__ : LUT filename: /software/geo2grid_v_1_1/pyspectral_data/rayleigh_only/rayleigh_lut_us-standard.h5 [2024-11-13 16:26:31,376] : PID 150 : DEBUG : pyspectral.rsr_reader : _get_filename : Filename: /software/geo2grid_v_1_1/pyspectral_data/rsr_abi_GOES-18.h5 [2024-11-13 16:26:31,377] : PID 150 : DEBUG : pyspectral.rsr_reader : _check_filename_exist : Filename: /software/geo2grid_v_1_1/pyspectral_data/rsr_abi_GOES-18.h5 [2024-11-13 16:26:31,378] : PID 150 : DEBUG : h5py._conv : __getitem__ : Creating converter from 3 to 5 [2024-11-13 16:26:31,379] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,381] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,383] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,385] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,387] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,389] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,391] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,393] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,395] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,397] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,399] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,401] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,403] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,405] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,406] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,408] : PID 150 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 16:26:31,412] : PID 150 : DEBUG : pyspectral.rayleigh : _get_effective_wavelength_and_band_name : Band name: ch2 Effective wavelength: 0.635057um [2024-11-13 16:26:31,415] : PID 150 : ERROR : abi_l1b_geotiff : exc_handler : Unexpected error. Enable debug messages (-vvv) or see log file for details. [2024-11-13 16:26:31,415] : PID 150 : DEBUG : abi_l1b_geotiff : exc_handler : Unexpected error exception: Traceback (most recent call last): File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 494, in sys.exit(main()) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 277, in main ret = processor() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 331, in __call__ return self._run_processing() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 362, in _run_processing scn.generate_possible_composites(True) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1417, in generate_possible_composites keepables = self._generate_composites_from_loaded_datasets() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1436, in _generate_composites_from_loaded_datasets return self._generate_composites_nodes_from_loaded_datasets(needed_comp_nodes) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1442, in _generate_composites_nodes_from_loaded_datasets self._generate_composite(node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1466, in _generate_composite prereq_datasets = self._get_prereq_datasets( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1548, in _get_prereq_datasets self._generate_composite(prereq_node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1500, in _generate_composite composite = compositor(prereq_datasets, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/modifiers/atmosphere.py", line 71, in __call__ refl_cor_band = corrector.get_reflectance(sunz, satz, ssadiff, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 270, in get_reflectance res = _map_blocks_or_direct_call(self._relax_rayleigh_refl_correction_where_cloudy, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 59, in _map_blocks_or_direct_call return da.map_blocks(func, *args, **kwargs) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/core.py", line 873, in map_blocks out = blockwise( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/blockwise.py", line 269, in blockwise raise ValueError( ValueError: Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks [2024-11-13 16:26:31,418] : PID 150 : ERROR : traceback : exc_handler : Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks Traceback (most recent call last): File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 494, in sys.exit(main()) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 277, in main ret = processor() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 331, in __call__ return self._run_processing() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 362, in _run_processing scn.generate_possible_composites(True) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1417, in generate_possible_composites keepables = self._generate_composites_from_loaded_datasets() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1436, in _generate_composites_from_loaded_datasets return self._generate_composites_nodes_from_loaded_datasets(needed_comp_nodes) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1442, in _generate_composites_nodes_from_loaded_datasets self._generate_composite(node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1466, in _generate_composite prereq_datasets = self._get_prereq_datasets( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1548, in _get_prereq_datasets self._generate_composite(prereq_node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1500, in _generate_composite composite = compositor(prereq_datasets, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/modifiers/atmosphere.py", line 71, in __call__ refl_cor_band = corrector.get_reflectance(sunz, satz, ssadiff, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 270, in get_reflectance res = _map_blocks_or_direct_call(self._relax_rayleigh_refl_correction_where_cloudy, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 59, in _map_blocks_or_direct_call return da.map_blocks(func, *args, **kwargs) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/core.py", line 873, in map_blocks out = blockwise( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/blockwise.py", line 269, in blockwise raise ValueError( ValueError: Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks