[2024-11-13 06:15:23,161] : DEBUG : abi_l1b_geotiff : _prepare_initial_logging : Starting script with arguments: -r abi_l1b -w geotiff -p true_color_night -f /input/OR_ABI-L1b-RadC-M6C01_G16_s20243180606152_e20243180608525_c20243180608579.nc /input/OR_ABI-L1b-RadC-M6C02_G16_s20243180606152_e20243180608525_c20243180608555.nc /input/OR_ABI-L1b-RadC-M6C02_G16_s20243180611152_e20243180613525_c20243180613555.nc /input/OR_ABI-L1b-RadC-M6C14_G16_s20243180606152_e20243180608525_c20243180609020.nc -vv [2024-11-13 06:15:23,162] : INFO : abi_l1b_geotiff : _run_processing : Sorting and reading input files... [2024-11-13 06:15:23,162] : DEBUG : abi_l1b_geotiff : _set_preferred_chunk_size : Setting preferred chunk size to 1356 pixels or 14MiB [2024-11-13 06:15:23,172] : DEBUG : satpy.readers.yaml_reader : load_yaml_configs : Reading ('/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/etc/readers/abi_l1b.yaml',) [2024-11-13 06:15:23,266] : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to abi_l1b: ['/input/OR_ABI-L1b-RadC-M6C01_G16_s20243180606152_e20243180608525_c20243180608579.nc', '/input/OR_ABI-L1b-RadC-M6C02_G16_s20243180611152_e20243180613525_c20243180613555.nc', '/input/OR_ABI-L1b-RadC-M6C02_G16_s20243180606152_e20243180608525_c20243180608555.nc', '/input/OR_ABI-L1b-RadC-M6C14_G16_s20243180606152_e20243180608525_c20243180609020.nc'] [2024-11-13 06:15:23,503] : PID 68 : DEBUG : polar2grid.core.script_utils : rename_log_file : Log renamed from '/output/abi_l1b_geotiff_fail.log' to 'abi_l1b_geotiff_20241113_060615.log' [2024-11-13 06:15:23,504] : PID 68 : INFO : abi_l1b_geotiff : _run_processing : Loading product metadata from files... [2024-11-13 06:15:23,504] : PID 68 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file abi.yaml [2024-11-13 06:15:24,363] : PID 68 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file visir.yaml [2024-11-13 06:15:24,448] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,456] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_azimuth_angle', calibration='reflectance') [2024-11-13 06:15:24,456] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,457] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_azimuth_angle', calibration='reflectance') [2024-11-13 06:15:24,457] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,461] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,469] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_azimuth_angle', calibration='reflectance') [2024-11-13 06:15:24,470] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='satellite_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='satellite_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,470] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_azimuth_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_azimuth_angle', calibration='reflectance') [2024-11-13 06:15:24,471] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,474] : PID 68 : DEBUG : satpy.node : _create_optional_subtrees : Skipping optional DataQuery(name='solar_zenith_angle', calibration='reflectance'): Unknown dataset DataQuery(name='solar_zenith_angle', calibration='reflectance') [2024-11-13 06:15:24,477] : PID 68 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C02. [2024-11-13 06:15:24,562] : PID 68 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C02. [2024-11-13 06:15:24,637] : PID 68 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C14. [2024-11-13 06:15:24,715] : PID 68 : WARNING : satpy.readers.yaml_reader : _get_file_handlers : Required file type 'c03' not found or loaded for 'C03' [2024-11-13 06:15:24,715] : PID 68 : DEBUG : satpy.readers.abi_l1b : get_dataset : Reading in get_dataset C01. [2024-11-13 06:15:24,778] : PID 68 : DEBUG : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DataID(name='C03', wavelength=WavelengthRange(min=0.8455, central=0.865, max=0.8845, unit='µm'), resolution=1000, calibration=, modifiers=('sunz_corrected',))': 'DataID(name='C03', wavelength=WavelengthRange(min=0.8455, central=0.865, max=0.8845, unit='µm'), resolution=1000, calibration=, modifiers=())' [2024-11-13 06:15:24,779] : PID 68 : DEBUG : satpy.modifiers.geometry : __call__ : Applying sun zen correction [2024-11-13 06:15:24,780] : PID 68 : DEBUG : satpy.modifiers.geometry : __call__ : Computing sun zenith angles. [2024-11-13 06:15:24,795] : PID 68 : DEBUG : satpy.modifiers.geometry : _apply_correction : Apply the standard sun-zenith correction [1/cos(sunz)] [2024-11-13 06:15:24,814] : PID 68 : DEBUG : h5py._conv : : Creating converter from 7 to 5 [2024-11-13 06:15:24,815] : PID 68 : DEBUG : h5py._conv : : Creating converter from 5 to 7 [2024-11-13 06:15:24,815] : PID 68 : DEBUG : h5py._conv : : Creating converter from 7 to 5 [2024-11-13 06:15:24,816] : PID 68 : DEBUG : h5py._conv : : Creating converter from 5 to 7 [2024-11-13 06:15:24,971] : PID 68 : INFO : satpy.modifiers.atmosphere : __call__ : Removing Rayleigh scattering with atmosphere 'us-standard' and aerosol type 'rayleigh_only' for 'C02' [2024-11-13 06:15:24,973] : PID 68 : INFO : pyspectral.rayleigh : __init__ : Atmosphere chosen: us-standard [2024-11-13 06:15:24,975] : PID 68 : DEBUG : pyspectral.rayleigh : __init__ : LUT filename: /software/geo2grid_v_1_1/pyspectral_data/rayleigh_only/rayleigh_lut_us-standard.h5 [2024-11-13 06:15:24,976] : PID 68 : DEBUG : pyspectral.rsr_reader : _get_filename : Filename: /software/geo2grid_v_1_1/pyspectral_data/rsr_abi_GOES-16.h5 [2024-11-13 06:15:24,977] : PID 68 : DEBUG : pyspectral.rsr_reader : _check_filename_exist : Filename: /software/geo2grid_v_1_1/pyspectral_data/rsr_abi_GOES-16.h5 [2024-11-13 06:15:24,980] : PID 68 : DEBUG : h5py._conv : __getitem__ : Creating converter from 3 to 5 [2024-11-13 06:15:24,980] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,983] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,984] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,986] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,988] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,990] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,992] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,994] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,996] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,997] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:24,999] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,001] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,003] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,004] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,006] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,008] : PID 68 : DEBUG : pyspectral.rsr_reader : get_number_of_detectors4bandname : No detectors found - assume only one... [2024-11-13 06:15:25,013] : PID 68 : DEBUG : pyspectral.rayleigh : _get_effective_wavelength_and_band_name : Band name: ch2 Effective wavelength: 0.635429um [2024-11-13 06:15:25,024] : PID 68 : ERROR : abi_l1b_geotiff : exc_handler : Unexpected error. Enable debug messages (-vvv) or see log file for details. [2024-11-13 06:15:25,025] : PID 68 : DEBUG : abi_l1b_geotiff : exc_handler : Unexpected error exception: Traceback (most recent call last): File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 494, in sys.exit(main()) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 277, in main ret = processor() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 331, in __call__ return self._run_processing() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 362, in _run_processing scn.generate_possible_composites(True) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1417, in generate_possible_composites keepables = self._generate_composites_from_loaded_datasets() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1436, in _generate_composites_from_loaded_datasets return self._generate_composites_nodes_from_loaded_datasets(needed_comp_nodes) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1442, in _generate_composites_nodes_from_loaded_datasets self._generate_composite(node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1466, in _generate_composite prereq_datasets = self._get_prereq_datasets( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1548, in _get_prereq_datasets self._generate_composite(prereq_node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1500, in _generate_composite composite = compositor(prereq_datasets, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/modifiers/atmosphere.py", line 71, in __call__ refl_cor_band = corrector.get_reflectance(sunz, satz, ssadiff, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 270, in get_reflectance res = _map_blocks_or_direct_call(self._relax_rayleigh_refl_correction_where_cloudy, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 59, in _map_blocks_or_direct_call return da.map_blocks(func, *args, **kwargs) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/core.py", line 873, in map_blocks out = blockwise( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/blockwise.py", line 269, in blockwise raise ValueError( ValueError: Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks [2024-11-13 06:15:25,054] : PID 68 : ERROR : traceback : exc_handler : Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks Traceback (most recent call last): File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 494, in sys.exit(main()) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 277, in main ret = processor() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 331, in __call__ return self._run_processing() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/polar2grid/glue.py", line 362, in _run_processing scn.generate_possible_composites(True) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1417, in generate_possible_composites keepables = self._generate_composites_from_loaded_datasets() File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1436, in _generate_composites_from_loaded_datasets return self._generate_composites_nodes_from_loaded_datasets(needed_comp_nodes) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1442, in _generate_composites_nodes_from_loaded_datasets self._generate_composite(node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1466, in _generate_composite prereq_datasets = self._get_prereq_datasets( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1548, in _get_prereq_datasets self._generate_composite(prereq_node, keepables) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/scene.py", line 1500, in _generate_composite composite = compositor(prereq_datasets, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/satpy/modifiers/atmosphere.py", line 71, in __call__ refl_cor_band = corrector.get_reflectance(sunz, satz, ssadiff, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 270, in get_reflectance res = _map_blocks_or_direct_call(self._relax_rayleigh_refl_correction_where_cloudy, File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/pyspectral/rayleigh.py", line 59, in _map_blocks_or_direct_call return da.map_blocks(func, *args, **kwargs) File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/core.py", line 873, in map_blocks out = blockwise( File "/software/geo2grid_v_1_1/libexec/python_runtime/lib/python3.10/site-packages/dask/array/blockwise.py", line 269, in blockwise raise ValueError( ValueError: Dimension 0 has 10 blocks, adjust_chunks specified with 2 blocks