Number of AVX cores detected in PC: 8
AVX compilation speedup in PC     : 1
Target device                     : AM62A
PYTHONPATH                        : .:
TIDL_TOOLS_PATH                   : ../edgeai-benchmark/tools/AM62A/tidl_tools
LD_LIBRARY_PATH                   : ../edgeai-benchmark/tools/AM62A/tidl_tools
argv: ['./scripts/run_modelmaker.py', 'config_detection_cuda_Shirley.yaml', '--target_device', 'AM62A']
run_modelmaker.py !!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!
{'common': {'target_module': 'vision', 'task_type': 'detection', 'target_device': 'AM62A', 'run_name': '{date-time}/{model_name}'}, 'dataset': {'enable': True, 'dataset_name': 'Shirley-20250319', 'input_data_path': '/local_data/home/mattmak/self_driving.v4i.coco.zip'}, 'training': {'enable': True, 'find_unused_parameters': True, 'pretrained': True, 'freeze_layers': ['backbone'], 'num_workers': 4, 'model_name': 'yolox_s_lite', 'training_epochs': 100, 'batch_size': 16, 'learning_rate': 0.005, 'num_gpus': 1, 'warmup_epochs': 5}, 'compilation': {'enable': True, 'tensor_bits': 8, 'calibration_dataset': False}}
<class 'dict'>
!!!!!!!!!!!!!!!!!!!!!!
{'common': {'verbose_mode': True, 'download_path': './data/downloads', 'projects_path': './data/projects', 'project_path': None, 'project_run_path': None, 'task_type': 'detection', 'target_machine': 'evm', 'target_device': 'AM62A', 'run_name': '{date-time}/{model_name}', 'target_module': 'vision'}, 'download': [{'download_url': 'https://software-dl.ti.com/jacinto7/esd/modelzoo/08_06_00_01/models/vision/detection/coco/edgeai-mmdet/yolox_s_lite_640x640_20220221_checkpoint.pth', 'download_path': '{download_path}/pretrained/yolox_s_lite'}], 'dataset': {'enable': True, 'dataset_name': 'Shirley-20250319', 'dataset_path': None, 'extract_path': None, 'split_factor': 0.8, 'split_names': ('train', 'val'), 'max_num_files': 10000, 'input_data_path': '/local_data/home/mattmak/self_driving.v4i.coco.zip', 'input_annotation_path': None, 'data_path_splits': None, 'data_dir': 'images', 'annotation_path_splits': None, 'annotation_dir': 'annotations', 'annotation_prefix': 'instances', 'annotation_format': 'coco_json', 'dataset_download': True, 'dataset_reload': False}, 'training': {'enable': True, 'model_name': 'yolox_s_lite', 'model_training_id': 'yolox_s_lite', 'training_backend': 'edgeai_mmdetection', 'pretrained_checkpoint_path': {'download_url': 'https://software-dl.ti.com/jacinto7/esd/modelzoo/08_06_00_01/models/vision/detection/coco/edgeai-mmdet/yolox_s_lite_640x640_20220221_checkpoint.pth', 'download_path': '{download_path}/pretrained/yolox_s_lite'}, 'pretrained_weight_state_dict_name': None, 'target_devices': {'TDA4VM': {'performance_fps': None, 'performance_infer_time_ms': 10.14, 'accuracy_factor': 56.9, 'accuracy_unit': 'AP50%', 'accuracy_factor2': 38.3, 'accuracy_unit2': 'AP[.5:.95]%'}, 'AM62A': {'performance_fps': None, 'performance_infer_time_ms': 43.94, 'accuracy_factor': 56.9, 'accuracy_unit': 'AP50%', 'accuracy_factor2': 38.3, 'accuracy_unit2': 'AP[.5:.95]%'}, 'AM67A': {'performance_fps': None, 'performance_infer_time_ms': '43.94 (with 1/2 device capability)', 'accuracy_factor': 56.9, 'accuracy_unit': 'AP50%', 'accuracy_factor2': 38.3, 'accuracy_unit2': 'AP[.5:.95]%'}, 'AM68A': {'performance_fps': None, 'performance_infer_time_ms': 10.22, 'accuracy_factor': 56.9, 'accuracy_unit': 'AP50%', 'accuracy_factor2': 38.3, 'accuracy_unit2': 'AP[.5:.95]%'}, 'AM69A': {'performance_fps': None, 'performance_infer_time_ms': '9.82 (with 1/4th device capability)', 'accuracy_factor': 56.9, 'accuracy_unit': 'AP50%', 'accuracy_factor2': 38.3, 'accuracy_unit2': 'AP[.5:.95]%'}}, 'project_path': None, 'dataset_path': None, 'training_path': None, 'log_file_path': None, 'log_summary_regex': None, 'summary_file_path': None, 'model_checkpoint_path': None, 'model_export_path': None, 'model_proto_path': None, 'model_packaged_path': None, 'training_epochs': 100, 'warmup_epochs': 5, 'num_last_epochs': 5, 'batch_size': 16, 'learning_rate': 0.005, 'num_classes': None, 'weight_decay': 0.0001, 'input_resize': 640, 'input_cropsize': 640, 'training_device': None, 'num_gpus': 1, 'distributed': True, 'training_master_port': 29500, 'with_background_class': None, 'model_architecture': 'yolox', 'training_devices': {'cpu': True, 'cuda': True}, 'find_unused_parameters': True, 'pretrained': True, 'freeze_layers': ['backbone'], 'num_workers': 4}, 'compilation': {'enable': True, 'preset_name': None, 'model_compilation_id': 'od-8220', 'compilation_path': None, 'model_compiled_path': None, 'log_file_path': None, 'log_summary_regex': None, 'summary_file_path': None, 'output_tensors_path': None, 'model_packaged_path': None, 'model_visualization_path': None, 'tensor_bits': 8, 'calibration_frames': 10, 'calibration_iterations': 10, 'num_frames': None, 'num_output_frames': 50, 'detection_threshold': 0.6, 'detection_top_k': 200, 'save_output': True, 'tidl_offload': True, 'input_optimization': False, 'capture_log': True, 'runtime_options': {'advanced_options:output_feature_16bit_names_list': '/multi_level_conv_obj.2/Conv_output_0, /multi_level_conv_reg.2/Conv_output_0, /multi_level_conv_cls.2/Conv_output_0, /multi_level_conv_obj.1/Conv_output_0, /multi_level_conv_reg.1/Conv_output_0, /multi_level_conv_cls.1/Conv_output_0, /multi_level_conv_obj.0/Conv_output_0, /multi_level_conv_reg.0/Conv_output_0, /multi_level_conv_cls.0/Conv_output_0'}, 'metric': {'label_offset_pred': 0}, 'calibration_dataset': False}}
---------------------------------------------------------------------
Run Name: 20250319-174853/yolox_s_lite
- Model: yolox_s_lite
- TargetDevices & Estimated Inference Times (ms): {'TDA4VM': 10.14, 'AM62A': 43.94, 'AM67A': '43.94 (with 1/2 device capability)', 'AM68A': 10.22, 'AM69A': '9.82 (with 1/4th device capability)'}
- This model can be compiled for the above device(s).
---------------------------------------------------------------------
assuming the given download_url is a valid path: /local_data/home/mattmak/self_driving.v4i.coco.zip
dataset split sizes {'train': 31281, 'val': 7974}
max_num_files is set to: 10000
dataset split sizes are limited to: {'train': 8000, 'val': 2000}
dataset loading OK
loading annotations into memory...
Done (t=0.24s)
creating index...
index created!
loading annotations into memory...
Done (t=0.03s)
creating index...
index created!
Selecting model configs from Python module: ./configs
Run params is at: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/run.yaml
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! detection.py: config_strs:  /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/edgeai_modelmaker/ai_modules/vision/training/edgeai_mmdetection/detection.py:447: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use-env is set by default in torchrun.
If your script expects `--local-rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  distributed_launch.main()
/local_data/home/mattmak/.pyenv/versions/benchmark_cuda/lib/python3.10/site-packages/mmengine/utils/dl_utils/setup_env.py:46: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  warnings.warn(
/local_data/home/mattmak/.pyenv/versions/benchmark_cuda/lib/python3.10/site-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  warnings.warn(
03/19 17:50:31 - mmengine - INFO - 
------------------------------------------------------------
System environment:
    sys.platform: linux
    Python: 3.10.16 (main, Feb  5 2025, 11:58:43) [GCC 11.4.0]
    CUDA available: True
    MUSA available: False
    numpy_random_seed: 1247723281
    GPU 0: NVIDIA GeForce RTX 2080 Ti
    CUDA_HOME: /usr/local/cuda-11.8
    NVCC: Cuda compilation tools, release 11.8, V11.8.89
    GCC: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
    PyTorch: 2.6.0+cu118
    PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201703
  - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.5.3 (Git Hash 66f0cb9eb66affd2da3bf5f8d897376f04aae6af)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90
  - CuDNN 90.1
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=2236df1770800ffea5697b11b0bb0d910b2e59e1, CUDA_VERSION=11.8, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.6.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 

    TorchVision: 0.21.0+cu118
    OpenCV: 4.11.0
    MMEngine: 0.10.5

Runtime environment:
    cudnn_benchmark: False
    mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}
    dist_cfg: {'backend': 'nccl'}
    seed: 1247723281
    Distributed launcher: pytorch
    Distributed training: True
    GPU number: 1
------------------------------------------------------------

03/19 17:50:33 - mmengine - INFO - Config:
auto_scale_lr = dict(base_batch_size=64, enable=False)
backend_args = None
base_lr = 0.01
classes = (
    'objects',
    'biker',
    'car',
    'pedestrian',
    'trafficLight',
    'trafficLight-Green',
    'trafficLight-GreenLeft',
    'trafficLight-Red',
    'trafficLight-RedLeft',
    'trafficLight-Yellow',
    'trafficLight-YellowLeft',
    'truck',
)
convert_to_lite_model = dict(model_surgery=1)
custom_hooks = [
    dict(num_last_epochs=15, priority=48, type='YOLOXModeSwitchHook'),
    dict(priority=48, type='SyncNormHook'),
    dict(
        ema_type='ExpMomentumEMA',
        momentum=0.0001,
        priority=49,
        type='EMAHook',
        update_buffers=True),
]
data_root = '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset'
dataset_type = 'CocoDataset'
default_hooks = dict(
    checkpoint=dict(interval=1, max_keep_ckpts=3, type='CheckpointHook'),
    logger=dict(interval=50, type='LoggerHook'),
    param_scheduler=dict(type='ParamSchedulerHook'),
    sampler_seed=dict(type='DistSamplerSeedHook'),
    timer=dict(type='IterTimerHook'),
    visualization=dict(type='DetVisualizationHook'))
default_scope = 'mmdet'
env_cfg = dict(
    cudnn_benchmark=False,
    dist_cfg=dict(backend='nccl'),
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0))
export_onnx_model = True
find_unused_parameters = True
img_scale = (
    640,
    640,
)
img_scales = [
    (
        640,
        640,
    ),
    (
        320,
        320,
    ),
    (
        960,
        960,
    ),
]
interval = 1
launcher = 'pytorch'
load_from = './data/downloads/pretrained/yolox_s_lite/yolox_s_lite_640x640_20220221_checkpoint.pth'
log_level = 'INFO'
log_processor = dict(by_epoch=True, type='LogProcessor', window_size=50)
max_epochs = 100
model = dict(
    backbone=dict(
        act_cfg=dict(type='ReLU'),
        deepen_factor=0.33,
        norm_cfg=dict(eps=0.001, momentum=0.03, type='BN'),
        out_indices=(
            2,
            3,
            4,
        ),
        spp_kernal_sizes=(
            5,
            9,
            13,
        ),
        type='CSPDarknet',
        use_depthwise=False,
        widen_factor=0.5),
    bbox_head=dict(
        act_cfg=dict(type='ReLU'),
        feat_channels=128,
        in_channels=128,
        loss_bbox=dict(
            eps=1e-16,
            loss_weight=5.0,
            mode='square',
            reduction='sum',
            type='IoULoss'),
        loss_cls=dict(
            loss_weight=1.0,
            reduction='sum',
            type='CrossEntropyLoss',
            use_sigmoid=True),
        loss_l1=dict(loss_weight=1.0, reduction='sum', type='L1Loss'),
        loss_obj=dict(
            loss_weight=1.0,
            reduction='sum',
            type='CrossEntropyLoss',
            use_sigmoid=True),
        norm_cfg=dict(eps=0.001, momentum=0.03, type='BN'),
        num_classes=12,
        stacked_convs=2,
        strides=(
            8,
            16,
            32,
        ),
        type='YOLOXHead',
        use_depthwise=False),
    data_preprocessor=dict(
        batch_augments=[
            dict(
                interval=10,
                random_size_range=(
                    480,
                    800,
                ),
                size_divisor=32,
                type='BatchSyncRandomResize'),
        ],
        pad_size_divisor=32,
        type='DetDataPreprocessor'),
    neck=dict(
        act_cfg=dict(type='ReLU'),
        in_channels=[
            128,
            256,
            512,
        ],
        norm_cfg=dict(eps=0.001, momentum=0.03, type='BN'),
        num_csp_blocks=1,
        out_channels=128,
        type='YOLOXPAFPN',
        upsample_cfg=dict(mode='nearest', scale_factor=2),
        use_depthwise=False),
    test_cfg=dict(nms=dict(iou_threshold=0.65, type='nms'), score_thr=0.01),
    train_cfg=dict(assigner=dict(center_radius=2.5, type='SimOTAAssigner')),
    type='YOLOX')
num_last_epochs = 15
optim_wrapper = dict(
    optimizer=dict(
        lr=0.005, momentum=0.9, nesterov=True, type='SGD',
        weight_decay=0.0005),
    paramwise_cfg=dict(bias_decay_mult=0.0, norm_decay_mult=0.0),
    type='OptimWrapper')
param_scheduler = [
    dict(
        begin=0,
        by_epoch=True,
        convert_to_iter_based=True,
        end=5,
        type='mmdet.QuadraticWarmupLR'),
    dict(
        T_max=15,
        begin=5,
        by_epoch=True,
        convert_to_iter_based=True,
        end=15,
        eta_min=0.0005,
        type='CosineAnnealingLR'),
    dict(begin=15, by_epoch=True, end=30, factor=1, type='ConstantLR'),
]
quantization = 0
resume = False
test_cfg = dict(type='TestLoop')
test_dataloader = dict(
    batch_size=8,
    dataset=dict(
        ann_file=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json',
        backend_args=None,
        data_prefix=dict(img='val/'),
        data_root=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset',
        metainfo=dict(
            classes=(
                'objects',
                'biker',
                'car',
                'pedestrian',
                'trafficLight',
                'trafficLight-Green',
                'trafficLight-GreenLeft',
                'trafficLight-Red',
                'trafficLight-RedLeft',
                'trafficLight-Yellow',
                'trafficLight-YellowLeft',
                'truck',
            )),
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(keep_ratio=True, scale=(
                640,
                640,
            ), type='Resize'),
            dict(
                pad_to_square=True,
                pad_val=dict(img=(
                    114.0,
                    114.0,
                    114.0,
                )),
                type='Pad'),
            dict(type='LoadAnnotations', with_bbox=True),
            dict(
                meta_keys=(
                    'img_id',
                    'img_path',
                    'ori_shape',
                    'img_shape',
                    'scale_factor',
                ),
                type='PackDetInputs'),
        ],
        test_mode=True,
        type='CocoDataset'),
    drop_last=False,
    num_workers=4,
    persistent_workers=True,
    sampler=dict(shuffle=False, type='DefaultSampler'))
test_evaluator = dict(
    ann_file=
    '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json',
    backend_args=None,
    metric='bbox',
    type='CocoMetric')
test_pipeline = [
    dict(backend_args=None, type='LoadImageFromFile'),
    dict(keep_ratio=True, scale=(
        640,
        640,
    ), type='Resize'),
    dict(
        pad_to_square=True,
        pad_val=dict(img=(
            114.0,
            114.0,
            114.0,
        )),
        type='Pad'),
    dict(type='LoadAnnotations', with_bbox=True),
    dict(
        meta_keys=(
            'img_id',
            'img_path',
            'ori_shape',
            'img_shape',
            'scale_factor',
        ),
        type='PackDetInputs'),
]
train_cfg = dict(max_epochs=100, type='EpochBasedTrainLoop', val_interval=1)
train_dataloader = dict(
    batch_size=8,
    dataset=dict(
        dataset=dict(
            ann_file=
            '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_train.json',
            backend_args=None,
            data_prefix=dict(img='train/'),
            data_root=
            '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset',
            filter_cfg=dict(filter_empty_gt=False, min_size=32),
            metainfo=dict(
                classes=(
                    'objects',
                    'biker',
                    'car',
                    'pedestrian',
                    'trafficLight',
                    'trafficLight-Green',
                    'trafficLight-GreenLeft',
                    'trafficLight-Red',
                    'trafficLight-RedLeft',
                    'trafficLight-Yellow',
                    'trafficLight-YellowLeft',
                    'truck',
                )),
            pipeline=[
                dict(backend_args=None, type='LoadImageFromFile'),
                dict(type='LoadAnnotations', with_bbox=True),
            ],
            type='CocoDataset'),
        pipeline=[
            dict(img_scale=(
                640,
                640,
            ), pad_val=114.0, type='Mosaic'),
            dict(
                border=(
                    -320,
                    -320,
                ),
                scaling_ratio_range=(
                    0.1,
                    2,
                ),
                type='RandomAffine'),
            dict(
                img_scale=(
                    640,
                    640,
                ),
                pad_val=114.0,
                ratio_range=(
                    0.8,
                    1.6,
                ),
                type='MixUp'),
            dict(type='YOLOXHSVRandomAug'),
            dict(prob=0.5, type='RandomFlip'),
            dict(keep_ratio=True, scale=(
                640,
                640,
            ), type='Resize'),
            dict(
                pad_to_square=True,
                pad_val=dict(img=(
                    114.0,
                    114.0,
                    114.0,
                )),
                type='Pad'),
            dict(
                keep_empty=False,
                min_gt_bbox_wh=(
                    1,
                    1,
                ),
                type='FilterAnnotations'),
            dict(type='PackDetInputs'),
        ],
        type='MultiImageMixDataset'),
    num_workers=4,
    persistent_workers=True,
    sampler=dict(shuffle=True, type='DefaultSampler'))
train_dataset = dict(
    dataset=dict(
        ann_file=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_train.json',
        backend_args=None,
        data_prefix=dict(img='train/'),
        data_root=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset',
        filter_cfg=dict(filter_empty_gt=False, min_size=32),
        metainfo=dict(
            classes=(
                'objects',
                'biker',
                'car',
                'pedestrian',
                'trafficLight',
                'trafficLight-Green',
                'trafficLight-GreenLeft',
                'trafficLight-Red',
                'trafficLight-RedLeft',
                'trafficLight-Yellow',
                'trafficLight-YellowLeft',
                'truck',
            )),
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(type='LoadAnnotations', with_bbox=True),
        ],
        type='CocoDataset'),
    pipeline=[
        dict(img_scale=(
            640,
            640,
        ), pad_val=114.0, type='Mosaic'),
        dict(
            border=(
                -320,
                -320,
            ),
            scaling_ratio_range=(
                0.1,
                2,
            ),
            type='RandomAffine'),
        dict(
            img_scale=(
                640,
                640,
            ),
            pad_val=114.0,
            ratio_range=(
                0.8,
                1.6,
            ),
            type='MixUp'),
        dict(type='YOLOXHSVRandomAug'),
        dict(prob=0.5, type='RandomFlip'),
        dict(keep_ratio=True, scale=(
            640,
            640,
        ), type='Resize'),
        dict(
            pad_to_square=True,
            pad_val=dict(img=(
                114.0,
                114.0,
                114.0,
            )),
            type='Pad'),
        dict(
            keep_empty=False,
            min_gt_bbox_wh=(
                1,
                1,
            ),
            type='FilterAnnotations'),
        dict(type='PackDetInputs'),
    ],
    type='MultiImageMixDataset')
train_pipeline = [
    dict(img_scale=(
        640,
        640,
    ), pad_val=114.0, type='Mosaic'),
    dict(
        border=(
            -320,
            -320,
        ),
        scaling_ratio_range=(
            0.1,
            2,
        ),
        type='RandomAffine'),
    dict(
        img_scale=(
            640,
            640,
        ),
        pad_val=114.0,
        ratio_range=(
            0.8,
            1.6,
        ),
        type='MixUp'),
    dict(type='YOLOXHSVRandomAug'),
    dict(prob=0.5, type='RandomFlip'),
    dict(keep_ratio=True, scale=(
        640,
        640,
    ), type='Resize'),
    dict(
        pad_to_square=True,
        pad_val=dict(img=(
            114.0,
            114.0,
            114.0,
        )),
        type='Pad'),
    dict(keep_empty=False, min_gt_bbox_wh=(
        1,
        1,
    ), type='FilterAnnotations'),
    dict(type='PackDetInputs'),
]
tta_model = dict(
    tta_cfg=dict(max_per_img=100, nms=dict(iou_threshold=0.65, type='nms')),
    type='DetTTAModel')
tta_pipeline = [
    dict(backend_args=None, type='LoadImageFromFile'),
    dict(
        transforms=[
            [
                dict(keep_ratio=True, scale=(
                    640,
                    640,
                ), type='Resize'),
                dict(keep_ratio=True, scale=(
                    320,
                    320,
                ), type='Resize'),
                dict(keep_ratio=True, scale=(
                    960,
                    960,
                ), type='Resize'),
            ],
            [
                dict(prob=1.0, type='RandomFlip'),
                dict(prob=0.0, type='RandomFlip'),
            ],
            [
                dict(
                    pad_to_square=True,
                    pad_val=dict(img=(
                        114.0,
                        114.0,
                        114.0,
                    )),
                    type='Pad'),
            ],
            [
                dict(type='LoadAnnotations', with_bbox=True),
            ],
            [
                dict(
                    meta_keys=(
                        'img_id',
                        'img_path',
                        'ori_shape',
                        'img_shape',
                        'scale_factor',
                        'flip',
                        'flip_direction',
                    ),
                    type='PackDetInputs'),
            ],
        ],
        type='TestTimeAug'),
]
val_cfg = dict(type='ValLoop')
val_dataloader = dict(
    batch_size=8,
    dataset=dict(
        ann_file=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json',
        backend_args=None,
        data_prefix=dict(img='val/'),
        data_root=
        '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset',
        metainfo=dict(
            classes=(
                'objects',
                'biker',
                'car',
                'pedestrian',
                'trafficLight',
                'trafficLight-Green',
                'trafficLight-GreenLeft',
                'trafficLight-Red',
                'trafficLight-RedLeft',
                'trafficLight-Yellow',
                'trafficLight-YellowLeft',
                'truck',
            )),
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(keep_ratio=True, scale=(
                640,
                640,
            ), type='Resize'),
            dict(
                pad_to_square=True,
                pad_val=dict(img=(
                    114.0,
                    114.0,
                    114.0,
                )),
                type='Pad'),
            dict(type='LoadAnnotations', with_bbox=True),
            dict(
                meta_keys=(
                    'img_id',
                    'img_path',
                    'ori_shape',
                    'img_shape',
                    'scale_factor',
                ),
                type='PackDetInputs'),
        ],
        test_mode=True,
        type='CocoDataset'),
    drop_last=False,
    num_workers=4,
    persistent_workers=True,
    sampler=dict(shuffle=False, type='DefaultSampler'))
val_evaluator = dict(
    ann_file=
    '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json',
    backend_args=None,
    metric='bbox',
    type='CocoMetric')
vis_backends = [
    dict(type='LocalVisBackend'),
]
visualizer = dict(
    name='visualizer',
    type='DetLocalVisualizer',
    vis_backends=[
        dict(type='LocalVisBackend'),
    ])
work_dir = '/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training'

03/19 17:50:36 - mmengine - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH   ) RuntimeInfoHook                    
(49          ) EMAHook                            
(BELOW_NORMAL) LoggerHook                         
 -------------------- 
after_load_checkpoint:
(49          ) EMAHook                            
 -------------------- 
before_train:
(VERY_HIGH   ) RuntimeInfoHook                    
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
(VERY_LOW    ) CheckpointHook                     
 -------------------- 
before_train_epoch:
(VERY_HIGH   ) RuntimeInfoHook                    
(48          ) YOLOXModeSwitchHook                
(NORMAL      ) IterTimerHook                      
(NORMAL      ) DistSamplerSeedHook                
 -------------------- 
before_train_iter:
(VERY_HIGH   ) RuntimeInfoHook                    
(NORMAL      ) IterTimerHook                      
 -------------------- 
after_train_iter:
(VERY_HIGH   ) RuntimeInfoHook                    
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
(BELOW_NORMAL) LoggerHook                         
(LOW         ) ParamSchedulerHook                 
(VERY_LOW    ) CheckpointHook                     
 -------------------- 
after_train_epoch:
(NORMAL      ) IterTimerHook                      
(LOW         ) ParamSchedulerHook                 
(VERY_LOW    ) CheckpointHook                     
 -------------------- 
before_val:
(VERY_HIGH   ) RuntimeInfoHook                    
 -------------------- 
before_val_epoch:
(48          ) SyncNormHook                       
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
 -------------------- 
before_val_iter:
(NORMAL      ) IterTimerHook                      
 -------------------- 
after_val_iter:
(NORMAL      ) IterTimerHook                      
(NORMAL      ) DetVisualizationHook               
(BELOW_NORMAL) LoggerHook                         
 -------------------- 
after_val_epoch:
(VERY_HIGH   ) RuntimeInfoHook                    
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
(BELOW_NORMAL) LoggerHook                         
(LOW         ) ParamSchedulerHook                 
(VERY_LOW    ) CheckpointHook                     
 -------------------- 
after_val:
(VERY_HIGH   ) RuntimeInfoHook                    
 -------------------- 
before_save_checkpoint:
(49          ) EMAHook                            
 -------------------- 
after_train:
(VERY_HIGH   ) RuntimeInfoHook                    
(VERY_LOW    ) CheckpointHook                     
 -------------------- 
before_test:
(VERY_HIGH   ) RuntimeInfoHook                    
 -------------------- 
before_test_epoch:
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
 -------------------- 
before_test_iter:
(NORMAL      ) IterTimerHook                      
 -------------------- 
after_test_iter:
(NORMAL      ) IterTimerHook                      
(NORMAL      ) DetVisualizationHook               
(BELOW_NORMAL) LoggerHook                         
 -------------------- 
after_test_epoch:
(VERY_HIGH   ) RuntimeInfoHook                    
(49          ) EMAHook                            
(NORMAL      ) IterTimerHook                      
(BELOW_NORMAL) LoggerHook                         
 -------------------- 
after_test:
(VERY_HIGH   ) RuntimeInfoHook                    
 -------------------- 
after_run:
(BELOW_NORMAL) LoggerHook                         
 -------------------- 
/local_data/home/mattmak/edgeai-tensorlab/edgeai-modeloptimization/torchmodelopt/edgeai_torchmodelopt/xmodelopt/surgery/v1/__init__.py:68: UserWarning: WARNING - xmodelopt.v1.surgery can only replace modules. To replace functions or operators, please use the torch.fx based xmodelopt.v2.surgery instead
  warnings.warn("WARNING - xmodelopt.v1.surgery can only replace modules. To replace functions or operators, please use the torch.fx based xmodelopt.v2.surgery instead")
model surgery done
loading annotations into memory...
Done (t=0.23s)
creating index...
index created!
03/19 17:50:39 - mmengine - INFO - paramwise_options -- backbone.stem.conv_in.bn.weight:weight_decay=0.0
03/19 17:50:39 - mmengine - INFO - paramwise_options -- backbone.stem.conv_in.bn.bias:weight_decay=0.0
03/19 17:50:39 - mmengine - INFO - paramwise_options -- backbone.stem.conv.bn.weight:weight_decay=0.0
03/19 17:50:39 - mmengine - INFO - paramwise_options -- backbone.stem.conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage1.1.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.1.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.1.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.1.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.1.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.2.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.2.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.2.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage2.1.blocks.2.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.1.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.1.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.1.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.1.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.2.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.2.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.2.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage3.1.blocks.2.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.1.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.1.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.1.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.1.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- backbone.stage4.2.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.reduce_layers.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.reduce_layers.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.reduce_layers.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.reduce_layers.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.0.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.top_down_blocks.1.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.downsamples.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.downsamples.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.downsamples.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.downsamples.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.0.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.main_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.main_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.short_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.short_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.final_conv.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.final_conv.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.blocks.0.conv1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.blocks.0.conv1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.blocks.0.conv2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.bottom_up_blocks.1.blocks.0.conv2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.2.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- neck.out_convs.2.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.0.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.0.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.0.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.0.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.1.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.1.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.1.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.1.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.2.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.2.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.2.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_cls_convs.2.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.0.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.0.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.0.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.0.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.1.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.1.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.1.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.1.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.2.0.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.2.0.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.2.1.bn.weight:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_reg_convs.2.1.bn.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_cls.0.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_cls.1.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_cls.2.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_reg.0.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_reg.1.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_reg.2.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_obj.0.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_obj.1.bias:weight_decay=0.0
03/19 17:50:40 - mmengine - INFO - paramwise_options -- bbox_head.multi_level_conv_obj.2.bias:weight_decay=0.0
loading annotations into memory...
Done (t=0.03s)
creating index...
index created!
loading annotations into memory...
Done (t=0.03s)
creating index...
index created!
03/19 17:50:42 - mmengine - WARNING - init_weights of YOLOX has been called more than once.
Loads checkpoint by local backend from path: ./data/downloads/pretrained/yolox_s_lite/yolox_s_lite_640x640_20220221_checkpoint.pth
The model and loaded state dict do not match exactly

size mismatch for bbox_head.multi_level_conv_cls.0.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.0.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
size mismatch for bbox_head.multi_level_conv_cls.1.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.1.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
size mismatch for bbox_head.multi_level_conv_cls.2.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.2.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
unexpected key in source state_dict: ema_backbone_stem_conv_in_conv_weight, ema_backbone_stem_conv_in_bn_weight, ema_backbone_stem_conv_in_bn_bias, ema_backbone_stem_conv_in_bn_running_mean, ema_backbone_stem_conv_in_bn_running_var, ema_backbone_stem_conv_in_bn_num_batches_tracked, ema_backbone_stem_conv_conv_weight, ema_backbone_stem_conv_bn_weight, ema_backbone_stem_conv_bn_bias, ema_backbone_stem_conv_bn_running_mean, ema_backbone_stem_conv_bn_running_var, ema_backbone_stem_conv_bn_num_batches_tracked, ema_backbone_stage1_0_conv_weight, ema_backbone_stage1_0_bn_weight, ema_backbone_stage1_0_bn_bias, ema_backbone_stage1_0_bn_running_mean, ema_backbone_stage1_0_bn_running_var, ema_backbone_stage1_0_bn_num_batches_tracked, ema_backbone_stage1_1_main_conv_conv_weight, ema_backbone_stage1_1_main_conv_bn_weight, ema_backbone_stage1_1_main_conv_bn_bias, ema_backbone_stage1_1_main_conv_bn_running_mean, ema_backbone_stage1_1_main_conv_bn_running_var, ema_backbone_stage1_1_main_conv_bn_num_batches_tracked, ema_backbone_stage1_1_short_conv_conv_weight, ema_backbone_stage1_1_short_conv_bn_weight, ema_backbone_stage1_1_short_conv_bn_bias, ema_backbone_stage1_1_short_conv_bn_running_mean, ema_backbone_stage1_1_short_conv_bn_running_var, ema_backbone_stage1_1_short_conv_bn_num_batches_tracked, ema_backbone_stage1_1_final_conv_conv_weight, ema_backbone_stage1_1_final_conv_bn_weight, ema_backbone_stage1_1_final_conv_bn_bias, ema_backbone_stage1_1_final_conv_bn_running_mean, ema_backbone_stage1_1_final_conv_bn_running_var, ema_backbone_stage1_1_final_conv_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_conv1_conv_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_bias, ema_backbone_stage1_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv1_bn_running_var, ema_backbone_stage1_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_conv2_conv_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_bias, ema_backbone_stage1_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv2_bn_running_var, ema_backbone_stage1_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage2_0_conv_weight, ema_backbone_stage2_0_bn_weight, ema_backbone_stage2_0_bn_bias, ema_backbone_stage2_0_bn_running_mean, ema_backbone_stage2_0_bn_running_var, ema_backbone_stage2_0_bn_num_batches_tracked, ema_backbone_stage2_1_main_conv_conv_weight, ema_backbone_stage2_1_main_conv_bn_weight, ema_backbone_stage2_1_main_conv_bn_bias, ema_backbone_stage2_1_main_conv_bn_running_mean, ema_backbone_stage2_1_main_conv_bn_running_var, ema_backbone_stage2_1_main_conv_bn_num_batches_tracked, ema_backbone_stage2_1_short_conv_conv_weight, ema_backbone_stage2_1_short_conv_bn_weight, ema_backbone_stage2_1_short_conv_bn_bias, ema_backbone_stage2_1_short_conv_bn_running_mean, ema_backbone_stage2_1_short_conv_bn_running_var, ema_backbone_stage2_1_short_conv_bn_num_batches_tracked, ema_backbone_stage2_1_final_conv_conv_weight, ema_backbone_stage2_1_final_conv_bn_weight, ema_backbone_stage2_1_final_conv_bn_bias, ema_backbone_stage2_1_final_conv_bn_running_mean, ema_backbone_stage2_1_final_conv_bn_running_var, ema_backbone_stage2_1_final_conv_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_conv1_conv_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_bias, ema_backbone_stage2_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv1_bn_running_var, ema_backbone_stage2_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_conv2_conv_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_bias, ema_backbone_stage2_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv2_bn_running_var, ema_backbone_stage2_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv1_conv_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_bias, ema_backbone_stage2_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv1_bn_running_var, ema_backbone_stage2_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv2_conv_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_bias, ema_backbone_stage2_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv2_bn_running_var, ema_backbone_stage2_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv1_conv_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_bias, ema_backbone_stage2_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv1_bn_running_var, ema_backbone_stage2_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv2_conv_weight, ema_backbone_stage2_1_blocks_2_conv2_bn_weight, ema_backbone_stage2_1_blocks_2_conv2_bn_bias, ema_backbone_stage2_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv2_bn_running_var, ema_backbone_stage2_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage3_0_conv_weight, ema_backbone_stage3_0_bn_weight, ema_backbone_stage3_0_bn_bias, ema_backbone_stage3_0_bn_running_mean, ema_backbone_stage3_0_bn_running_var, ema_backbone_stage3_0_bn_num_batches_tracked, ema_backbone_stage3_1_main_conv_conv_weight, ema_backbone_stage3_1_main_conv_bn_weight, ema_backbone_stage3_1_main_conv_bn_bias, ema_backbone_stage3_1_main_conv_bn_running_mean, ema_backbone_stage3_1_main_conv_bn_running_var, ema_backbone_stage3_1_main_conv_bn_num_batches_tracked, ema_backbone_stage3_1_short_conv_conv_weight, ema_backbone_stage3_1_short_conv_bn_weight, ema_backbone_stage3_1_short_conv_bn_bias, ema_backbone_stage3_1_short_conv_bn_running_mean, ema_backbone_stage3_1_short_conv_bn_running_var, ema_backbone_stage3_1_short_conv_bn_num_batches_tracked, ema_backbone_stage3_1_final_conv_conv_weight, ema_backbone_stage3_1_final_conv_bn_weight, ema_backbone_stage3_1_final_conv_bn_bias, ema_backbone_stage3_1_final_conv_bn_running_mean, ema_backbone_stage3_1_final_conv_bn_running_var, ema_backbone_stage3_1_final_conv_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv1_conv_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_bias, ema_backbone_stage3_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv1_bn_running_var, ema_backbone_stage3_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv2_conv_weight, ema_backbone_stage3_1_blocks_0_conv2_bn_weight, ema_backbone_stage3_1_blocks_0_conv2_bn_bias, ema_backbone_stage3_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv2_bn_running_var, ema_backbone_stage3_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv1_conv_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_bias, ema_backbone_stage3_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_1_conv1_bn_running_var, ema_backbone_stage3_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv2_conv_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_bias, ema_backbone_stage3_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_1_conv2_bn_running_var, ema_backbone_stage3_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_2_conv1_conv_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_bias, ema_backbone_stage3_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv1_bn_running_var, ema_backbone_stage3_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_2_conv2_conv_weight, ema_backbone_stage3_1_blocks_2_conv2_bn_weight, ema_backbone_stage3_1_blocks_2_conv2_bn_bias, ema_backbone_stage3_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv2_bn_running_var, ema_backbone_stage3_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage4_0_conv_weight, ema_backbone_stage4_0_bn_weight, ema_backbone_stage4_0_bn_bias, ema_backbone_stage4_0_bn_running_mean, ema_backbone_stage4_0_bn_running_var, ema_backbone_stage4_0_bn_num_batches_tracked, ema_backbone_stage4_1_conv1_conv_weight, ema_backbone_stage4_1_conv1_bn_weight, ema_backbone_stage4_1_conv1_bn_bias, ema_backbone_stage4_1_conv1_bn_running_mean, ema_backbone_stage4_1_conv1_bn_running_var, ema_backbone_stage4_1_conv1_bn_num_batches_tracked, ema_backbone_stage4_1_conv2_conv_weight, ema_backbone_stage4_1_conv2_bn_weight, ema_backbone_stage4_1_conv2_bn_bias, ema_backbone_stage4_1_conv2_bn_running_mean, ema_backbone_stage4_1_conv2_bn_running_var, ema_backbone_stage4_1_conv2_bn_num_batches_tracked, ema_backbone_stage4_2_main_conv_conv_weight, ema_backbone_stage4_2_main_conv_bn_weight, ema_backbone_stage4_2_main_conv_bn_bias, ema_backbone_stage4_2_main_conv_bn_running_mean, ema_backbone_stage4_2_main_conv_bn_running_var, ema_backbone_stage4_2_main_conv_bn_num_batches_tracked, ema_backbone_stage4_2_short_conv_conv_weight, ema_backbone_stage4_2_short_conv_bn_weight, ema_backbone_stage4_2_short_conv_bn_bias, ema_backbone_stage4_2_short_conv_bn_running_mean, ema_backbone_stage4_2_short_conv_bn_running_var, ema_backbone_stage4_2_short_conv_bn_num_batches_tracked, ema_backbone_stage4_2_final_conv_conv_weight, ema_backbone_stage4_2_final_conv_bn_weight, ema_backbone_stage4_2_final_conv_bn_bias, ema_backbone_stage4_2_final_conv_bn_running_mean, ema_backbone_stage4_2_final_conv_bn_running_var, ema_backbone_stage4_2_final_conv_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv1_conv_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_bias, ema_backbone_stage4_2_blocks_0_conv1_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv1_bn_running_var, ema_backbone_stage4_2_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv2_conv_weight, ema_backbone_stage4_2_blocks_0_conv2_bn_weight, ema_backbone_stage4_2_blocks_0_conv2_bn_bias, ema_backbone_stage4_2_blocks_0_conv2_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv2_bn_running_var, ema_backbone_stage4_2_blocks_0_conv2_bn_num_batches_tracked, ema_neck_reduce_layers_0_conv_weight, ema_neck_reduce_layers_0_bn_weight, ema_neck_reduce_layers_0_bn_bias, ema_neck_reduce_layers_0_bn_running_mean, ema_neck_reduce_layers_0_bn_running_var, ema_neck_reduce_layers_0_bn_num_batches_tracked, ema_neck_reduce_layers_1_conv_weight, ema_neck_reduce_layers_1_bn_weight, ema_neck_reduce_layers_1_bn_bias, ema_neck_reduce_layers_1_bn_running_mean, ema_neck_reduce_layers_1_bn_running_var, ema_neck_reduce_layers_1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_main_conv_conv_weight, ema_neck_top_down_blocks_0_main_conv_bn_weight, ema_neck_top_down_blocks_0_main_conv_bn_bias, ema_neck_top_down_blocks_0_main_conv_bn_running_mean, ema_neck_top_down_blocks_0_main_conv_bn_running_var, ema_neck_top_down_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_short_conv_conv_weight, ema_neck_top_down_blocks_0_short_conv_bn_weight, ema_neck_top_down_blocks_0_short_conv_bn_bias, ema_neck_top_down_blocks_0_short_conv_bn_running_mean, ema_neck_top_down_blocks_0_short_conv_bn_running_var, ema_neck_top_down_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_final_conv_conv_weight, ema_neck_top_down_blocks_0_final_conv_bn_weight, ema_neck_top_down_blocks_0_final_conv_bn_bias, ema_neck_top_down_blocks_0_final_conv_bn_running_mean, ema_neck_top_down_blocks_0_final_conv_bn_running_var, ema_neck_top_down_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_0_conv2_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_1_main_conv_conv_weight, ema_neck_top_down_blocks_1_main_conv_bn_weight, ema_neck_top_down_blocks_1_main_conv_bn_bias, ema_neck_top_down_blocks_1_main_conv_bn_running_mean, ema_neck_top_down_blocks_1_main_conv_bn_running_var, ema_neck_top_down_blocks_1_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_short_conv_conv_weight, ema_neck_top_down_blocks_1_short_conv_bn_weight, ema_neck_top_down_blocks_1_short_conv_bn_bias, ema_neck_top_down_blocks_1_short_conv_bn_running_mean, ema_neck_top_down_blocks_1_short_conv_bn_running_var, ema_neck_top_down_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_final_conv_conv_weight, ema_neck_top_down_blocks_1_final_conv_bn_weight, ema_neck_top_down_blocks_1_final_conv_bn_bias, ema_neck_top_down_blocks_1_final_conv_bn_running_mean, ema_neck_top_down_blocks_1_final_conv_bn_running_var, ema_neck_top_down_blocks_1_final_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv2_conv_weight, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_downsamples_0_conv_weight, ema_neck_downsamples_0_bn_weight, ema_neck_downsamples_0_bn_bias, ema_neck_downsamples_0_bn_running_mean, ema_neck_downsamples_0_bn_running_var, ema_neck_downsamples_0_bn_num_batches_tracked, ema_neck_downsamples_1_conv_weight, ema_neck_downsamples_1_bn_weight, ema_neck_downsamples_1_bn_bias, ema_neck_downsamples_1_bn_running_mean, ema_neck_downsamples_1_bn_running_var, ema_neck_downsamples_1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_main_conv_conv_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_bias, ema_neck_bottom_up_blocks_0_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_main_conv_bn_running_var, ema_neck_bottom_up_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_short_conv_conv_weight, ema_neck_bottom_up_blocks_0_short_conv_bn_weight, ema_neck_bottom_up_blocks_0_short_conv_bn_bias, ema_neck_bottom_up_blocks_0_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_short_conv_bn_running_var, ema_neck_bottom_up_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_final_conv_conv_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_bias, ema_neck_bottom_up_blocks_0_final_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_final_conv_bn_running_var, ema_neck_bottom_up_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_main_conv_conv_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_bias, ema_neck_bottom_up_blocks_1_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_main_conv_bn_running_var, ema_neck_bottom_up_blocks_1_main_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_short_conv_conv_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_bias, ema_neck_bottom_up_blocks_1_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_short_conv_bn_running_var, ema_neck_bottom_up_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_final_conv_conv_weight, ema_neck_bottom_up_blocks_1_final_conv_bn_weight, ema_neck_bottom_up_blocks_1_final_conv_bn_bias, ema_neck_bottom_up_blocks_1_final_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_final_conv_bn_running_var, ema_neck_bottom_up_blocks_1_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_out_convs_0_conv_weight, ema_neck_out_convs_0_bn_weight, ema_neck_out_convs_0_bn_bias, ema_neck_out_convs_0_bn_running_mean, ema_neck_out_convs_0_bn_running_var, ema_neck_out_convs_0_bn_num_batches_tracked, ema_neck_out_convs_1_conv_weight, ema_neck_out_convs_1_bn_weight, ema_neck_out_convs_1_bn_bias, ema_neck_out_convs_1_bn_running_mean, ema_neck_out_convs_1_bn_running_var, ema_neck_out_convs_1_bn_num_batches_tracked, ema_neck_out_convs_2_conv_weight, ema_neck_out_convs_2_bn_weight, ema_neck_out_convs_2_bn_bias, ema_neck_out_convs_2_bn_running_mean, ema_neck_out_convs_2_bn_running_var, ema_neck_out_convs_2_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_0_0_conv_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_bias, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_0_1_conv_weight, ema_bbox_head_multi_level_cls_convs_0_1_bn_weight, ema_bbox_head_multi_level_cls_convs_0_1_bn_bias, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_0_conv_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_bias, ema_bbox_head_multi_level_cls_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_1_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_1_conv_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_bias, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_2_0_conv_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_bias, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_2_1_conv_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_bias, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_0_conv_weight, ema_bbox_head_multi_level_reg_convs_0_0_bn_weight, ema_bbox_head_multi_level_reg_convs_0_0_bn_bias, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_1_conv_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_bias, ema_bbox_head_multi_level_reg_convs_0_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_0_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_0_conv_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_bias, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_1_conv_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_bias, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_0_conv_weight, ema_bbox_head_multi_level_reg_convs_2_0_bn_weight, ema_bbox_head_multi_level_reg_convs_2_0_bn_bias, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_1_conv_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_bias, ema_bbox_head_multi_level_reg_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_conv_cls_0_weight, ema_bbox_head_multi_level_conv_cls_0_bias, ema_bbox_head_multi_level_conv_cls_1_weight, ema_bbox_head_multi_level_conv_cls_1_bias, ema_bbox_head_multi_level_conv_cls_2_weight, ema_bbox_head_multi_level_conv_cls_2_bias, ema_bbox_head_multi_level_conv_reg_0_weight, ema_bbox_head_multi_level_conv_reg_0_bias, ema_bbox_head_multi_level_conv_reg_1_weight, ema_bbox_head_multi_level_conv_reg_1_bias, ema_bbox_head_multi_level_conv_reg_2_weight, ema_bbox_head_multi_level_conv_reg_2_bias, ema_bbox_head_multi_level_conv_obj_0_weight, ema_bbox_head_multi_level_conv_obj_0_bias, ema_bbox_head_multi_level_conv_obj_1_weight, ema_bbox_head_multi_level_conv_obj_1_bias, ema_bbox_head_multi_level_conv_obj_2_weight, ema_bbox_head_multi_level_conv_obj_2_bias

The model and loaded state dict do not match exactly

size mismatch for bbox_head.multi_level_conv_cls.0.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.0.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
size mismatch for bbox_head.multi_level_conv_cls.1.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.1.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
size mismatch for bbox_head.multi_level_conv_cls.2.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.2.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([12]).
unexpected key in source state_dict: ema_backbone_stem_conv_in_conv_weight, ema_backbone_stem_conv_in_bn_weight, ema_backbone_stem_conv_in_bn_bias, ema_backbone_stem_conv_in_bn_running_mean, ema_backbone_stem_conv_in_bn_running_var, ema_backbone_stem_conv_in_bn_num_batches_tracked, ema_backbone_stem_conv_conv_weight, ema_backbone_stem_conv_bn_weight, ema_backbone_stem_conv_bn_bias, ema_backbone_stem_conv_bn_running_mean, ema_backbone_stem_conv_bn_running_var, ema_backbone_stem_conv_bn_num_batches_tracked, ema_backbone_stage1_0_conv_weight, ema_backbone_stage1_0_bn_weight, ema_backbone_stage1_0_bn_bias, ema_backbone_stage1_0_bn_running_mean, ema_backbone_stage1_0_bn_running_var, ema_backbone_stage1_0_bn_num_batches_tracked, ema_backbone_stage1_1_main_conv_conv_weight, ema_backbone_stage1_1_main_conv_bn_weight, ema_backbone_stage1_1_main_conv_bn_bias, ema_backbone_stage1_1_main_conv_bn_running_mean, ema_backbone_stage1_1_main_conv_bn_running_var, ema_backbone_stage1_1_main_conv_bn_num_batches_tracked, ema_backbone_stage1_1_short_conv_conv_weight, ema_backbone_stage1_1_short_conv_bn_weight, ema_backbone_stage1_1_short_conv_bn_bias, ema_backbone_stage1_1_short_conv_bn_running_mean, ema_backbone_stage1_1_short_conv_bn_running_var, ema_backbone_stage1_1_short_conv_bn_num_batches_tracked, ema_backbone_stage1_1_final_conv_conv_weight, ema_backbone_stage1_1_final_conv_bn_weight, ema_backbone_stage1_1_final_conv_bn_bias, ema_backbone_stage1_1_final_conv_bn_running_mean, ema_backbone_stage1_1_final_conv_bn_running_var, ema_backbone_stage1_1_final_conv_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_conv1_conv_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_bias, ema_backbone_stage1_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv1_bn_running_var, ema_backbone_stage1_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_conv2_conv_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_bias, ema_backbone_stage1_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv2_bn_running_var, ema_backbone_stage1_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage2_0_conv_weight, ema_backbone_stage2_0_bn_weight, ema_backbone_stage2_0_bn_bias, ema_backbone_stage2_0_bn_running_mean, ema_backbone_stage2_0_bn_running_var, ema_backbone_stage2_0_bn_num_batches_tracked, ema_backbone_stage2_1_main_conv_conv_weight, ema_backbone_stage2_1_main_conv_bn_weight, ema_backbone_stage2_1_main_conv_bn_bias, ema_backbone_stage2_1_main_conv_bn_running_mean, ema_backbone_stage2_1_main_conv_bn_running_var, ema_backbone_stage2_1_main_conv_bn_num_batches_tracked, ema_backbone_stage2_1_short_conv_conv_weight, ema_backbone_stage2_1_short_conv_bn_weight, ema_backbone_stage2_1_short_conv_bn_bias, ema_backbone_stage2_1_short_conv_bn_running_mean, ema_backbone_stage2_1_short_conv_bn_running_var, ema_backbone_stage2_1_short_conv_bn_num_batches_tracked, ema_backbone_stage2_1_final_conv_conv_weight, ema_backbone_stage2_1_final_conv_bn_weight, ema_backbone_stage2_1_final_conv_bn_bias, ema_backbone_stage2_1_final_conv_bn_running_mean, ema_backbone_stage2_1_final_conv_bn_running_var, ema_backbone_stage2_1_final_conv_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_conv1_conv_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_bias, ema_backbone_stage2_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv1_bn_running_var, ema_backbone_stage2_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_conv2_conv_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_bias, ema_backbone_stage2_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv2_bn_running_var, ema_backbone_stage2_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv1_conv_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_bias, ema_backbone_stage2_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv1_bn_running_var, ema_backbone_stage2_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv2_conv_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_bias, ema_backbone_stage2_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv2_bn_running_var, ema_backbone_stage2_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv1_conv_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_bias, ema_backbone_stage2_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv1_bn_running_var, ema_backbone_stage2_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv2_conv_weight, ema_backbone_stage2_1_blocks_2_conv2_bn_weight, ema_backbone_stage2_1_blocks_2_conv2_bn_bias, ema_backbone_stage2_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv2_bn_running_var, ema_backbone_stage2_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage3_0_conv_weight, ema_backbone_stage3_0_bn_weight, ema_backbone_stage3_0_bn_bias, ema_backbone_stage3_0_bn_running_mean, ema_backbone_stage3_0_bn_running_var, ema_backbone_stage3_0_bn_num_batches_tracked, ema_backbone_stage3_1_main_conv_conv_weight, ema_backbone_stage3_1_main_conv_bn_weight, ema_backbone_stage3_1_main_conv_bn_bias, ema_backbone_stage3_1_main_conv_bn_running_mean, ema_backbone_stage3_1_main_conv_bn_running_var, ema_backbone_stage3_1_main_conv_bn_num_batches_tracked, ema_backbone_stage3_1_short_conv_conv_weight, ema_backbone_stage3_1_short_conv_bn_weight, ema_backbone_stage3_1_short_conv_bn_bias, ema_backbone_stage3_1_short_conv_bn_running_mean, ema_backbone_stage3_1_short_conv_bn_running_var, ema_backbone_stage3_1_short_conv_bn_num_batches_tracked, ema_backbone_stage3_1_final_conv_conv_weight, ema_backbone_stage3_1_final_conv_bn_weight, ema_backbone_stage3_1_final_conv_bn_bias, ema_backbone_stage3_1_final_conv_bn_running_mean, ema_backbone_stage3_1_final_conv_bn_running_var, ema_backbone_stage3_1_final_conv_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv1_conv_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_bias, ema_backbone_stage3_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv1_bn_running_var, ema_backbone_stage3_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv2_conv_weight, ema_backbone_stage3_1_blocks_0_conv2_bn_weight, ema_backbone_stage3_1_blocks_0_conv2_bn_bias, ema_backbone_stage3_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv2_bn_running_var, ema_backbone_stage3_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv1_conv_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_bias, ema_backbone_stage3_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_1_conv1_bn_running_var, ema_backbone_stage3_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv2_conv_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_bias, ema_backbone_stage3_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_1_conv2_bn_running_var, ema_backbone_stage3_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_2_conv1_conv_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_bias, ema_backbone_stage3_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv1_bn_running_var, ema_backbone_stage3_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_2_conv2_conv_weight, ema_backbone_stage3_1_blocks_2_conv2_bn_weight, ema_backbone_stage3_1_blocks_2_conv2_bn_bias, ema_backbone_stage3_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv2_bn_running_var, ema_backbone_stage3_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage4_0_conv_weight, ema_backbone_stage4_0_bn_weight, ema_backbone_stage4_0_bn_bias, ema_backbone_stage4_0_bn_running_mean, ema_backbone_stage4_0_bn_running_var, ema_backbone_stage4_0_bn_num_batches_tracked, ema_backbone_stage4_1_conv1_conv_weight, ema_backbone_stage4_1_conv1_bn_weight, ema_backbone_stage4_1_conv1_bn_bias, ema_backbone_stage4_1_conv1_bn_running_mean, ema_backbone_stage4_1_conv1_bn_running_var, ema_backbone_stage4_1_conv1_bn_num_batches_tracked, ema_backbone_stage4_1_conv2_conv_weight, ema_backbone_stage4_1_conv2_bn_weight, ema_backbone_stage4_1_conv2_bn_bias, ema_backbone_stage4_1_conv2_bn_running_mean, ema_backbone_stage4_1_conv2_bn_running_var, ema_backbone_stage4_1_conv2_bn_num_batches_tracked, ema_backbone_stage4_2_main_conv_conv_weight, ema_backbone_stage4_2_main_conv_bn_weight, ema_backbone_stage4_2_main_conv_bn_bias, ema_backbone_stage4_2_main_conv_bn_running_mean, ema_backbone_stage4_2_main_conv_bn_running_var, ema_backbone_stage4_2_main_conv_bn_num_batches_tracked, ema_backbone_stage4_2_short_conv_conv_weight, ema_backbone_stage4_2_short_conv_bn_weight, ema_backbone_stage4_2_short_conv_bn_bias, ema_backbone_stage4_2_short_conv_bn_running_mean, ema_backbone_stage4_2_short_conv_bn_running_var, ema_backbone_stage4_2_short_conv_bn_num_batches_tracked, ema_backbone_stage4_2_final_conv_conv_weight, ema_backbone_stage4_2_final_conv_bn_weight, ema_backbone_stage4_2_final_conv_bn_bias, ema_backbone_stage4_2_final_conv_bn_running_mean, ema_backbone_stage4_2_final_conv_bn_running_var, ema_backbone_stage4_2_final_conv_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv1_conv_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_bias, ema_backbone_stage4_2_blocks_0_conv1_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv1_bn_running_var, ema_backbone_stage4_2_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv2_conv_weight, ema_backbone_stage4_2_blocks_0_conv2_bn_weight, ema_backbone_stage4_2_blocks_0_conv2_bn_bias, ema_backbone_stage4_2_blocks_0_conv2_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv2_bn_running_var, ema_backbone_stage4_2_blocks_0_conv2_bn_num_batches_tracked, ema_neck_reduce_layers_0_conv_weight, ema_neck_reduce_layers_0_bn_weight, ema_neck_reduce_layers_0_bn_bias, ema_neck_reduce_layers_0_bn_running_mean, ema_neck_reduce_layers_0_bn_running_var, ema_neck_reduce_layers_0_bn_num_batches_tracked, ema_neck_reduce_layers_1_conv_weight, ema_neck_reduce_layers_1_bn_weight, ema_neck_reduce_layers_1_bn_bias, ema_neck_reduce_layers_1_bn_running_mean, ema_neck_reduce_layers_1_bn_running_var, ema_neck_reduce_layers_1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_main_conv_conv_weight, ema_neck_top_down_blocks_0_main_conv_bn_weight, ema_neck_top_down_blocks_0_main_conv_bn_bias, ema_neck_top_down_blocks_0_main_conv_bn_running_mean, ema_neck_top_down_blocks_0_main_conv_bn_running_var, ema_neck_top_down_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_short_conv_conv_weight, ema_neck_top_down_blocks_0_short_conv_bn_weight, ema_neck_top_down_blocks_0_short_conv_bn_bias, ema_neck_top_down_blocks_0_short_conv_bn_running_mean, ema_neck_top_down_blocks_0_short_conv_bn_running_var, ema_neck_top_down_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_final_conv_conv_weight, ema_neck_top_down_blocks_0_final_conv_bn_weight, ema_neck_top_down_blocks_0_final_conv_bn_bias, ema_neck_top_down_blocks_0_final_conv_bn_running_mean, ema_neck_top_down_blocks_0_final_conv_bn_running_var, ema_neck_top_down_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_0_conv2_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_1_main_conv_conv_weight, ema_neck_top_down_blocks_1_main_conv_bn_weight, ema_neck_top_down_blocks_1_main_conv_bn_bias, ema_neck_top_down_blocks_1_main_conv_bn_running_mean, ema_neck_top_down_blocks_1_main_conv_bn_running_var, ema_neck_top_down_blocks_1_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_short_conv_conv_weight, ema_neck_top_down_blocks_1_short_conv_bn_weight, ema_neck_top_down_blocks_1_short_conv_bn_bias, ema_neck_top_down_blocks_1_short_conv_bn_running_mean, ema_neck_top_down_blocks_1_short_conv_bn_running_var, ema_neck_top_down_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_final_conv_conv_weight, ema_neck_top_down_blocks_1_final_conv_bn_weight, ema_neck_top_down_blocks_1_final_conv_bn_bias, ema_neck_top_down_blocks_1_final_conv_bn_running_mean, ema_neck_top_down_blocks_1_final_conv_bn_running_var, ema_neck_top_down_blocks_1_final_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv2_conv_weight, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_downsamples_0_conv_weight, ema_neck_downsamples_0_bn_weight, ema_neck_downsamples_0_bn_bias, ema_neck_downsamples_0_bn_running_mean, ema_neck_downsamples_0_bn_running_var, ema_neck_downsamples_0_bn_num_batches_tracked, ema_neck_downsamples_1_conv_weight, ema_neck_downsamples_1_bn_weight, ema_neck_downsamples_1_bn_bias, ema_neck_downsamples_1_bn_running_mean, ema_neck_downsamples_1_bn_running_var, ema_neck_downsamples_1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_main_conv_conv_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_bias, ema_neck_bottom_up_blocks_0_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_main_conv_bn_running_var, ema_neck_bottom_up_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_short_conv_conv_weight, ema_neck_bottom_up_blocks_0_short_conv_bn_weight, ema_neck_bottom_up_blocks_0_short_conv_bn_bias, ema_neck_bottom_up_blocks_0_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_short_conv_bn_running_var, ema_neck_bottom_up_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_final_conv_conv_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_bias, ema_neck_bottom_up_blocks_0_final_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_final_conv_bn_running_var, ema_neck_bottom_up_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_main_conv_conv_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_bias, ema_neck_bottom_up_blocks_1_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_main_conv_bn_running_var, ema_neck_bottom_up_blocks_1_main_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_short_conv_conv_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_bias, ema_neck_bottom_up_blocks_1_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_short_conv_bn_running_var, ema_neck_bottom_up_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_final_conv_conv_weight, ema_neck_bottom_up_blocks_1_final_conv_bn_weight, ema_neck_bottom_up_blocks_1_final_conv_bn_bias, ema_neck_bottom_up_blocks_1_final_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_final_conv_bn_running_var, ema_neck_bottom_up_blocks_1_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_out_convs_0_conv_weight, ema_neck_out_convs_0_bn_weight, ema_neck_out_convs_0_bn_bias, ema_neck_out_convs_0_bn_running_mean, ema_neck_out_convs_0_bn_running_var, ema_neck_out_convs_0_bn_num_batches_tracked, ema_neck_out_convs_1_conv_weight, ema_neck_out_convs_1_bn_weight, ema_neck_out_convs_1_bn_bias, ema_neck_out_convs_1_bn_running_mean, ema_neck_out_convs_1_bn_running_var, ema_neck_out_convs_1_bn_num_batches_tracked, ema_neck_out_convs_2_conv_weight, ema_neck_out_convs_2_bn_weight, ema_neck_out_convs_2_bn_bias, ema_neck_out_convs_2_bn_running_mean, ema_neck_out_convs_2_bn_running_var, ema_neck_out_convs_2_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_0_0_conv_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_bias, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_0_1_conv_weight, ema_bbox_head_multi_level_cls_convs_0_1_bn_weight, ema_bbox_head_multi_level_cls_convs_0_1_bn_bias, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_0_conv_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_bias, ema_bbox_head_multi_level_cls_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_1_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_1_conv_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_bias, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_2_0_conv_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_bias, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_2_1_conv_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_bias, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_0_conv_weight, ema_bbox_head_multi_level_reg_convs_0_0_bn_weight, ema_bbox_head_multi_level_reg_convs_0_0_bn_bias, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_1_conv_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_bias, ema_bbox_head_multi_level_reg_convs_0_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_0_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_0_conv_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_bias, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_1_conv_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_bias, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_0_conv_weight, ema_bbox_head_multi_level_reg_convs_2_0_bn_weight, ema_bbox_head_multi_level_reg_convs_2_0_bn_bias, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_1_conv_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_bias, ema_bbox_head_multi_level_reg_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_conv_cls_0_weight, ema_bbox_head_multi_level_conv_cls_0_bias, ema_bbox_head_multi_level_conv_cls_1_weight, ema_bbox_head_multi_level_conv_cls_1_bias, ema_bbox_head_multi_level_conv_cls_2_weight, ema_bbox_head_multi_level_conv_cls_2_bias, ema_bbox_head_multi_level_conv_reg_0_weight, ema_bbox_head_multi_level_conv_reg_0_bias, ema_bbox_head_multi_level_conv_reg_1_weight, ema_bbox_head_multi_level_conv_reg_1_bias, ema_bbox_head_multi_level_conv_reg_2_weight, ema_bbox_head_multi_level_conv_reg_2_bias, ema_bbox_head_multi_level_conv_obj_0_weight, ema_bbox_head_multi_level_conv_obj_0_bias, ema_bbox_head_multi_level_conv_obj_1_weight, ema_bbox_head_multi_level_conv_obj_1_bias, ema_bbox_head_multi_level_conv_obj_2_weight, ema_bbox_head_multi_level_conv_obj_2_bias

03/19 17:50:43 - mmengine - INFO - Load checkpoint from ./data/downloads/pretrained/yolox_s_lite/yolox_s_lite_640x640_20220221_checkpoint.pth
03/19 17:50:43 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
03/19 17:50:43 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
03/19 17:50:43 - mmengine - INFO - Checkpoints will be saved to /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training.
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
/local_data/home/mattmak/.pyenv/versions/benchmark_cuda/lib/python3.10/site-packages/torch/functional.py:539: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:3637.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 17:51:07 - mmengine - INFO - Epoch(train)   [1][  50/1000]  base_lr: 5.0000e-07 lr: 5.0000e-07  eta: 13:18:52  time: 0.4796  data_time: 0.1304  memory: 3360  loss: 10.1158  loss_cls: 2.6341  loss_bbox: 3.0588  loss_obj: 4.4229
03/19 17:51:22 - mmengine - INFO - Epoch(train)   [1][ 100/1000]  base_lr: 2.0000e-06 lr: 2.0000e-06  eta: 10:42:43  time: 0.2925  data_time: 0.1001  memory: 3070  loss: 9.7866  loss_cls: 2.4926  loss_bbox: 3.0764  loss_obj: 4.2175
03/19 17:51:34 - mmengine - INFO - Epoch(train)   [1][ 150/1000]  base_lr: 4.5000e-06 lr: 4.5000e-06  eta: 9:25:12  time: 0.2469  data_time: 0.0264  memory: 3937  loss: 9.0935  loss_cls: 2.1273  loss_bbox: 3.0031  loss_obj: 3.9630
03/19 17:51:44 - mmengine - INFO - Epoch(train)   [1][ 200/1000]  base_lr: 8.0000e-06 lr: 8.0000e-06  eta: 8:28:18  time: 0.2035  data_time: 0.0092  memory: 3357  loss: 8.1595  loss_cls: 1.7489  loss_bbox: 2.9421  loss_obj: 3.4685
03/19 17:51:55 - mmengine - INFO - Epoch(train)   [1][ 250/1000]  base_lr: 1.2500e-05 lr: 1.2500e-05  eta: 7:55:48  time: 0.2086  data_time: 0.0086  memory: 2817  loss: 7.4578  loss_cls: 1.5312  loss_bbox: 2.8206  loss_obj: 3.1060
03/19 17:52:06 - mmengine - INFO - Epoch(train)   [1][ 300/1000]  base_lr: 1.8000e-05 lr: 1.8000e-05  eta: 7:37:45  time: 0.2219  data_time: 0.0085  memory: 3639  loss: 7.0876  loss_cls: 1.3660  loss_bbox: 2.7899  loss_obj: 2.9317
03/19 17:52:17 - mmengine - INFO - Epoch(train)   [1][ 350/1000]  base_lr: 2.4500e-05 lr: 2.4500e-05  eta: 7:25:55  time: 0.2266  data_time: 0.0085  memory: 3357  loss: 6.5590  loss_cls: 1.2085  loss_bbox: 2.7093  loss_obj: 2.6412
03/19 17:52:29 - mmengine - INFO - Epoch(train)   [1][ 400/1000]  base_lr: 3.2000e-05 lr: 3.2000e-05  eta: 7:17:23  time: 0.2285  data_time: 0.0083  memory: 3937  loss: 6.4474  loss_cls: 1.1779  loss_bbox: 2.6723  loss_obj: 2.5972
03/19 17:52:40 - mmengine - INFO - Epoch(train)   [1][ 450/1000]  base_lr: 4.0500e-05 lr: 4.0500e-05  eta: 7:11:49  time: 0.2345  data_time: 0.0084  memory: 3937  loss: 6.3446  loss_cls: 1.1310  loss_bbox: 2.7296  loss_obj: 2.4841
03/19 17:52:50 - mmengine - INFO - Epoch(train)   [1][ 500/1000]  base_lr: 5.0000e-05 lr: 5.0000e-05  eta: 7:00:46  time: 0.1949  data_time: 0.0085  memory: 2817  loss: 5.9983  loss_cls: 1.0532  loss_bbox: 2.6737  loss_obj: 2.2715
03/19 17:52:59 - mmengine - INFO - Epoch(train)   [1][ 550/1000]  base_lr: 6.0500e-05 lr: 6.0500e-05  eta: 6:50:19  time: 0.1858  data_time: 0.0085  memory: 2115  loss: 5.9374  loss_cls: 1.0412  loss_bbox: 2.7063  loss_obj: 2.1898
03/19 17:53:11 - mmengine - INFO - Epoch(train)   [1][ 600/1000]  base_lr: 7.2000e-05 lr: 7.2000e-05  eta: 6:48:44  time: 0.2376  data_time: 0.0083  memory: 3937  loss: 5.9249  loss_cls: 1.0227  loss_bbox: 2.6476  loss_obj: 2.2546
03/19 17:53:22 - mmengine - INFO - Epoch(train)   [1][ 650/1000]  base_lr: 8.4500e-05 lr: 8.4500e-05  eta: 6:44:50  time: 0.2177  data_time: 0.0083  memory: 3357  loss: 5.7609  loss_cls: 0.9931  loss_bbox: 2.6402  loss_obj: 2.1276
03/19 17:53:33 - mmengine - INFO - Epoch(train)   [1][ 700/1000]  base_lr: 9.8000e-05 lr: 9.8000e-05  eta: 6:41:15  time: 0.2158  data_time: 0.0085  memory: 3357  loss: 5.7169  loss_cls: 0.9854  loss_bbox: 2.6072  loss_obj: 2.1242
03/19 17:53:43 - mmengine - INFO - Epoch(train)   [1][ 750/1000]  base_lr: 1.1250e-04 lr: 1.1250e-04  eta: 6:37:15  time: 0.2080  data_time: 0.0082  memory: 2817  loss: 5.6854  loss_cls: 0.9567  loss_bbox: 2.6058  loss_obj: 2.1229
03/19 17:53:53 - mmengine - INFO - Epoch(train)   [1][ 800/1000]  base_lr: 1.2800e-04 lr: 1.2800e-04  eta: 6:33:13  time: 0.2031  data_time: 0.0086  memory: 3071  loss: 5.7057  loss_cls: 0.9717  loss_bbox: 2.6307  loss_obj: 2.1033
03/19 17:54:06 - mmengine - INFO - Epoch(train)   [1][ 850/1000]  base_lr: 1.4450e-04 lr: 1.4450e-04  eta: 6:33:42  time: 0.2449  data_time: 0.0083  memory: 3937  loss: 5.5777  loss_cls: 0.9346  loss_bbox: 2.5685  loss_obj: 2.0746
03/19 17:54:18 - mmengine - INFO - Epoch(train)   [1][ 900/1000]  base_lr: 1.6200e-04 lr: 1.6200e-04  eta: 6:34:58  time: 0.2541  data_time: 0.0083  memory: 3937  loss: 5.5403  loss_cls: 0.9185  loss_bbox: 2.5777  loss_obj: 2.0442
03/19 17:54:28 - mmengine - INFO - Epoch(train)   [1][ 950/1000]  base_lr: 1.8050e-04 lr: 1.8050e-04  eta: 6:30:38  time: 0.1917  data_time: 0.0084  memory: 2817  loss: 5.5314  loss_cls: 0.9264  loss_bbox: 2.6119  loss_obj: 1.9932
03/19 17:54:38 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 17:54:38 - mmengine - INFO - Epoch(train)   [1][1000/1000]  base_lr: 2.0000e-04 lr: 2.0000e-04  eta: 6:27:37  time: 0.2024  data_time: 0.0085  memory: 3357  loss: 5.4991  loss_cls: 0.9191  loss_bbox: 2.5688  loss_obj: 2.0112
03/19 17:54:38 - mmengine - INFO - Saving checkpoint at 1 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 17:54:45 - mmengine - INFO - Epoch(val)   [1][ 50/250]    eta: 0:00:17  time: 0.0882  data_time: 0.0301  memory: 527  
03/19 17:54:48 - mmengine - INFO - Epoch(val)   [1][100/250]    eta: 0:00:11  time: 0.0696  data_time: 0.0221  memory: 527  
03/19 17:54:52 - mmengine - INFO - Epoch(val)   [1][150/250]    eta: 0:00:07  time: 0.0671  data_time: 0.0202  memory: 527  
03/19 17:54:55 - mmengine - INFO - Epoch(val)   [1][200/250]    eta: 0:00:03  time: 0.0704  data_time: 0.0238  memory: 527  
03/19 17:54:58 - mmengine - INFO - Epoch(val)   [1][250/250]    eta: 0:00:00  time: 0.0656  data_time: 0.0190  memory: 527  
03/19 17:55:00 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.29s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.55s).
Accumulating evaluation results...
DONE (t=2.50s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.096
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.222
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.067
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.045
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.111
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.277
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.205
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.205
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.205
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.137
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.255
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.383
03/19 17:55:12 - mmengine - INFO - bbox_mAP_copypaste: 0.096 0.222 0.067 0.045 0.111 0.277
03/19 17:55:12 - mmengine - INFO - Epoch(val) [1][250/250]    coco/bbox_mAP: 0.0960  coco/bbox_mAP_50: 0.2220  coco/bbox_mAP_75: 0.0670  coco/bbox_mAP_s: 0.0450  coco/bbox_mAP_m: 0.1110  coco/bbox_mAP_l: 0.2770  data_time: 0.0230  time: 0.0722
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 17:55:23 - mmengine - INFO - Epoch(train)   [2][  50/1000]  base_lr: 2.2050e-04 lr: 2.2050e-04  eta: 6:26:45  time: 0.2265  data_time: 0.0188  memory: 3639  loss: 5.3675  loss_cls: 0.8854  loss_bbox: 2.5503  loss_obj: 1.9319
03/19 17:55:35 - mmengine - INFO - Epoch(train)   [2][ 100/1000]  base_lr: 2.4200e-04 lr: 2.4200e-04  eta: 6:26:31  time: 0.2339  data_time: 0.0084  memory: 3937  loss: 5.3353  loss_cls: 0.9000  loss_bbox: 2.5199  loss_obj: 1.9155
03/19 17:55:47 - mmengine - INFO - Epoch(train)   [2][ 150/1000]  base_lr: 2.6450e-04 lr: 2.6450e-04  eta: 6:26:06  time: 0.2313  data_time: 0.0085  memory: 3937  loss: 5.3587  loss_cls: 0.8864  loss_bbox: 2.5283  loss_obj: 1.9440
03/19 17:55:58 - mmengine - INFO - Epoch(train)   [2][ 200/1000]  base_lr: 2.8800e-04 lr: 2.8800e-04  eta: 6:25:02  time: 0.2217  data_time: 0.0084  memory: 3937  loss: 5.4114  loss_cls: 0.8802  loss_bbox: 2.5563  loss_obj: 1.9749
03/19 17:56:07 - mmengine - INFO - Epoch(train)   [2][ 250/1000]  base_lr: 3.1250e-04 lr: 3.1250e-04  eta: 6:22:12  time: 0.1937  data_time: 0.0086  memory: 3071  loss: 5.2565  loss_cls: 0.8727  loss_bbox: 2.5386  loss_obj: 1.8452
03/19 17:56:18 - mmengine - INFO - Epoch(train)   [2][ 300/1000]  base_lr: 3.3800e-04 lr: 3.3800e-04  eta: 6:20:47  time: 0.2130  data_time: 0.0085  memory: 3071  loss: 5.3000  loss_cls: 0.8685  loss_bbox: 2.5443  loss_obj: 1.8872
03/19 17:56:29 - mmengine - INFO - Epoch(train)   [2][ 350/1000]  base_lr: 3.6450e-04 lr: 3.6450e-04  eta: 6:20:10  time: 0.2244  data_time: 0.0087  memory: 3937  loss: 5.3086  loss_cls: 0.8655  loss_bbox: 2.5199  loss_obj: 1.9233
03/19 17:56:40 - mmengine - INFO - Epoch(train)   [2][ 400/1000]  base_lr: 3.9200e-04 lr: 3.9200e-04  eta: 6:18:25  time: 0.2047  data_time: 0.0084  memory: 3639  loss: 5.2766  loss_cls: 0.8687  loss_bbox: 2.5180  loss_obj: 1.8899
03/19 17:56:51 - mmengine - INFO - Epoch(train)   [2][ 450/1000]  base_lr: 4.2050e-04 lr: 4.2050e-04  eta: 6:18:25  time: 0.2337  data_time: 0.0085  memory: 3937  loss: 5.2074  loss_cls: 0.8447  loss_bbox: 2.5082  loss_obj: 1.8545
03/19 17:57:02 - mmengine - INFO - Epoch(train)   [2][ 500/1000]  base_lr: 4.5000e-04 lr: 4.5000e-04  eta: 6:17:17  time: 0.2134  data_time: 0.0084  memory: 3357  loss: 5.1977  loss_cls: 0.8454  loss_bbox: 2.5011  loss_obj: 1.8512
03/19 17:57:12 - mmengine - INFO - Epoch(train)   [2][ 550/1000]  base_lr: 4.8050e-04 lr: 4.8050e-04  eta: 6:15:13  time: 0.1941  data_time: 0.0086  memory: 3357  loss: 5.3019  loss_cls: 0.8730  loss_bbox: 2.5696  loss_obj: 1.8593
03/19 17:57:22 - mmengine - INFO - Epoch(train)   [2][ 600/1000]  base_lr: 5.1200e-04 lr: 5.1200e-04  eta: 6:14:20  time: 0.2154  data_time: 0.0085  memory: 3357  loss: 5.2338  loss_cls: 0.8410  loss_bbox: 2.5053  loss_obj: 1.8875
03/19 17:57:33 - mmengine - INFO - Epoch(train)   [2][ 650/1000]  base_lr: 5.4450e-04 lr: 5.4450e-04  eta: 6:13:06  time: 0.2070  data_time: 0.0086  memory: 3639  loss: 5.2378  loss_cls: 0.8474  loss_bbox: 2.5180  loss_obj: 1.8724
03/19 17:57:43 - mmengine - INFO - Epoch(train)   [2][ 700/1000]  base_lr: 5.7800e-04 lr: 5.7800e-04  eta: 6:12:01  time: 0.2094  data_time: 0.0084  memory: 3639  loss: 5.2048  loss_cls: 0.8470  loss_bbox: 2.5203  loss_obj: 1.8374
03/19 17:57:54 - mmengine - INFO - Epoch(train)   [2][ 750/1000]  base_lr: 6.1250e-04 lr: 6.1250e-04  eta: 6:11:04  time: 0.2108  data_time: 0.0085  memory: 2817  loss: 5.2050  loss_cls: 0.8468  loss_bbox: 2.5295  loss_obj: 1.8286
03/19 17:58:05 - mmengine - INFO - Epoch(train)   [2][ 800/1000]  base_lr: 6.4800e-04 lr: 6.4800e-04  eta: 6:11:06  time: 0.2313  data_time: 0.0083  memory: 3937  loss: 5.2819  loss_cls: 0.8355  loss_bbox: 2.5161  loss_obj: 1.9302
03/19 17:58:16 - mmengine - INFO - Epoch(train)   [2][ 850/1000]  base_lr: 6.8450e-04 lr: 6.8450e-04  eta: 6:10:20  time: 0.2136  data_time: 0.0086  memory: 3639  loss: 5.1940  loss_cls: 0.8420  loss_bbox: 2.5243  loss_obj: 1.8276
03/19 17:58:27 - mmengine - INFO - Epoch(train)   [2][ 900/1000]  base_lr: 7.2200e-04 lr: 7.2200e-04  eta: 6:10:12  time: 0.2278  data_time: 0.0086  memory: 3937  loss: 5.2840  loss_cls: 0.8450  loss_bbox: 2.5537  loss_obj: 1.8853
03/19 17:58:40 - mmengine - INFO - Epoch(train)   [2][ 950/1000]  base_lr: 7.6050e-04 lr: 7.6050e-04  eta: 6:10:46  time: 0.2443  data_time: 0.0086  memory: 3937  loss: 5.2989  loss_cls: 0.8383  loss_bbox: 2.5336  loss_obj: 1.9271
03/19 17:58:50 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 17:58:50 - mmengine - INFO - Epoch(train)   [2][1000/1000]  base_lr: 8.0000e-04 lr: 8.0000e-04  eta: 6:09:29  time: 0.2004  data_time: 0.0084  memory: 3937  loss: 5.2160  loss_cls: 0.8295  loss_bbox: 2.5282  loss_obj: 1.8583
03/19 17:58:50 - mmengine - INFO - Saving checkpoint at 2 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 17:58:55 - mmengine - INFO - Epoch(val)   [2][ 50/250]    eta: 0:00:11  time: 0.0589  data_time: 0.0070  memory: 527  
03/19 17:58:58 - mmengine - INFO - Epoch(val)   [2][100/250]    eta: 0:00:08  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 17:59:01 - mmengine - INFO - Epoch(val)   [2][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 17:59:04 - mmengine - INFO - Epoch(val)   [2][200/250]    eta: 0:00:02  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 17:59:07 - mmengine - INFO - Epoch(val)   [2][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0065  memory: 527  
03/19 17:59:09 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.33s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=10.61s).
Accumulating evaluation results...
DONE (t=3.34s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.116
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.283
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.079
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.063
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.133
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.275
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.229
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.229
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.229
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.164
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.263
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.414
03/19 17:59:24 - mmengine - INFO - bbox_mAP_copypaste: 0.116 0.283 0.079 0.063 0.133 0.275
03/19 17:59:24 - mmengine - INFO - Epoch(val) [2][250/250]    coco/bbox_mAP: 0.1160  coco/bbox_mAP_50: 0.2830  coco/bbox_mAP_75: 0.0790  coco/bbox_mAP_s: 0.0630  coco/bbox_mAP_m: 0.1330  coco/bbox_mAP_l: 0.2750  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 17:59:36 - mmengine - INFO - Epoch(train)   [3][  50/1000]  base_lr: 8.4050e-04 lr: 8.4050e-04  eta: 6:09:51  time: 0.2400  data_time: 0.0188  memory: 3937  loss: 5.1959  loss_cls: 0.8238  loss_bbox: 2.5205  loss_obj: 1.8516
03/19 17:59:46 - mmengine - INFO - Epoch(train)   [3][ 100/1000]  base_lr: 8.8200e-04 lr: 8.8200e-04  eta: 6:08:46  time: 0.2038  data_time: 0.0084  memory: 2817  loss: 5.1595  loss_cls: 0.8117  loss_bbox: 2.5144  loss_obj: 1.8333
03/19 17:59:58 - mmengine - INFO - Epoch(train)   [3][ 150/1000]  base_lr: 9.2450e-04 lr: 9.2450e-04  eta: 6:08:45  time: 0.2302  data_time: 0.0085  memory: 3937  loss: 5.1524  loss_cls: 0.8164  loss_bbox: 2.4984  loss_obj: 1.8376
03/19 18:00:08 - mmengine - INFO - Epoch(train)   [3][ 200/1000]  base_lr: 9.6800e-04 lr: 9.6800e-04  eta: 6:07:55  time: 0.2087  data_time: 0.0086  memory: 3937  loss: 5.2576  loss_cls: 0.8447  loss_bbox: 2.5683  loss_obj: 1.8447
03/19 18:00:17 - mmengine - INFO - Epoch(train)   [3][ 250/1000]  base_lr: 1.0125e-03 lr: 1.0125e-03  eta: 6:06:10  time: 0.1826  data_time: 0.0087  memory: 2587  loss: 5.1965  loss_cls: 0.8425  loss_bbox: 2.5773  loss_obj: 1.7766
03/19 18:00:28 - mmengine - INFO - Epoch(train)   [3][ 300/1000]  base_lr: 1.0580e-03 lr: 1.0580e-03  eta: 6:05:36  time: 0.2145  data_time: 0.0084  memory: 3937  loss: 5.1935  loss_cls: 0.8165  loss_bbox: 2.5429  loss_obj: 1.8341
03/19 18:00:40 - mmengine - INFO - Epoch(train)   [3][ 350/1000]  base_lr: 1.1045e-03 lr: 1.1045e-03  eta: 6:06:04  time: 0.2434  data_time: 0.0086  memory: 3937  loss: 5.1946  loss_cls: 0.8086  loss_bbox: 2.5009  loss_obj: 1.8851
03/19 18:00:51 - mmengine - INFO - Epoch(train)   [3][ 400/1000]  base_lr: 1.1520e-03 lr: 1.1520e-03  eta: 6:05:39  time: 0.2179  data_time: 0.0085  memory: 3937  loss: 5.1943  loss_cls: 0.8344  loss_bbox: 2.5291  loss_obj: 1.8307
03/19 18:01:02 - mmengine - INFO - Epoch(train)   [3][ 450/1000]  base_lr: 1.2005e-03 lr: 1.2005e-03  eta: 6:05:03  time: 0.2123  data_time: 0.0086  memory: 3071  loss: 5.2326  loss_cls: 0.8320  loss_bbox: 2.5480  loss_obj: 1.8525
03/19 18:01:11 - mmengine - INFO - Epoch(train)   [3][ 500/1000]  base_lr: 1.2500e-03 lr: 1.2500e-03  eta: 6:03:43  time: 0.1892  data_time: 0.0087  memory: 3071  loss: 5.2884  loss_cls: 0.8470  loss_bbox: 2.5875  loss_obj: 1.8539
03/19 18:01:22 - mmengine - INFO - Epoch(train)   [3][ 550/1000]  base_lr: 1.3005e-03 lr: 1.3005e-03  eta: 6:03:11  time: 0.2132  data_time: 0.0087  memory: 3937  loss: 5.2252  loss_cls: 0.8296  loss_bbox: 2.5390  loss_obj: 1.8566
03/19 18:01:33 - mmengine - INFO - Epoch(train)   [3][ 600/1000]  base_lr: 1.3520e-03 lr: 1.3520e-03  eta: 6:02:59  time: 0.2230  data_time: 0.0086  memory: 3937  loss: 5.2659  loss_cls: 0.8189  loss_bbox: 2.5491  loss_obj: 1.8979
03/19 18:01:44 - mmengine - INFO - Epoch(train)   [3][ 650/1000]  base_lr: 1.4045e-03 lr: 1.4045e-03  eta: 6:02:51  time: 0.2254  data_time: 0.0086  memory: 3937  loss: 5.2123  loss_cls: 0.8266  loss_bbox: 2.5373  loss_obj: 1.8484
03/19 18:01:55 - mmengine - INFO - Epoch(train)   [3][ 700/1000]  base_lr: 1.4580e-03 lr: 1.4580e-03  eta: 6:02:09  time: 0.2068  data_time: 0.0085  memory: 3071  loss: 5.1970  loss_cls: 0.8217  loss_bbox: 2.5513  loss_obj: 1.8240
03/19 18:02:05 - mmengine - INFO - Epoch(train)   [3][ 750/1000]  base_lr: 1.5125e-03 lr: 1.5125e-03  eta: 6:01:32  time: 0.2085  data_time: 0.0086  memory: 3357  loss: 5.1931  loss_cls: 0.8301  loss_bbox: 2.5586  loss_obj: 1.8044
03/19 18:02:16 - mmengine - INFO - Epoch(train)   [3][ 800/1000]  base_lr: 1.5680e-03 lr: 1.5680e-03  eta: 6:01:17  time: 0.2207  data_time: 0.0086  memory: 3639  loss: 5.1988  loss_cls: 0.8149  loss_bbox: 2.5466  loss_obj: 1.8373
03/19 18:02:27 - mmengine - INFO - Epoch(train)   [3][ 850/1000]  base_lr: 1.6245e-03 lr: 1.6245e-03  eta: 6:01:10  time: 0.2255  data_time: 0.0084  memory: 3937  loss: 5.2260  loss_cls: 0.8201  loss_bbox: 2.5479  loss_obj: 1.8579
03/19 18:02:39 - mmengine - INFO - Epoch(train)   [3][ 900/1000]  base_lr: 1.6820e-03 lr: 1.6820e-03  eta: 6:01:19  time: 0.2348  data_time: 0.0085  memory: 3937  loss: 5.2351  loss_cls: 0.8267  loss_bbox: 2.5539  loss_obj: 1.8545
03/19 18:02:50 - mmengine - INFO - Epoch(train)   [3][ 950/1000]  base_lr: 1.7405e-03 lr: 1.7405e-03  eta: 6:00:55  time: 0.2160  data_time: 0.0085  memory: 3357  loss: 5.2745  loss_cls: 0.8141  loss_bbox: 2.5589  loss_obj: 1.9015
03/19 18:03:02 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:03:02 - mmengine - INFO - Epoch(train)   [3][1000/1000]  base_lr: 1.8000e-03 lr: 1.8000e-03  eta: 6:00:55  time: 0.2300  data_time: 0.0083  memory: 3937  loss: 5.2131  loss_cls: 0.8066  loss_bbox: 2.5383  loss_obj: 1.8682
03/19 18:03:02 - mmengine - INFO - Saving checkpoint at 3 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:03:07 - mmengine - INFO - Epoch(val)   [3][ 50/250]    eta: 0:00:11  time: 0.0595  data_time: 0.0072  memory: 527  
03/19 18:03:10 - mmengine - INFO - Epoch(val)   [3][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0064  memory: 527  
03/19 18:03:13 - mmengine - INFO - Epoch(val)   [3][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 18:03:16 - mmengine - INFO - Epoch(val)   [3][200/250]    eta: 0:00:02  time: 0.0592  data_time: 0.0064  memory: 527  
03/19 18:03:19 - mmengine - INFO - Epoch(val)   [3][250/250]    eta: 0:00:00  time: 0.0567  data_time: 0.0063  memory: 527  
03/19 18:03:21 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.32s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.62s).
Accumulating evaluation results...
DONE (t=2.87s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.116
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.286
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.072
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.062
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.144
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.172
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.287
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.362
03/19 18:03:34 - mmengine - INFO - bbox_mAP_copypaste: 0.116 0.286 0.072 0.062 0.144 0.280
03/19 18:03:34 - mmengine - INFO - Epoch(val) [3][250/250]    coco/bbox_mAP: 0.1160  coco/bbox_mAP_50: 0.2860  coco/bbox_mAP_75: 0.0720  coco/bbox_mAP_s: 0.0620  coco/bbox_mAP_m: 0.1440  coco/bbox_mAP_l: 0.2800  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:03:46 - mmengine - INFO - Epoch(train)   [4][  50/1000]  base_lr: 1.8605e-03 lr: 1.8605e-03  eta: 6:01:00  time: 0.2333  data_time: 0.0193  memory: 3639  loss: 5.2293  loss_cls: 0.8124  loss_bbox: 2.5672  loss_obj: 1.8497
03/19 18:03:58 - mmengine - INFO - Epoch(train)   [4][ 100/1000]  base_lr: 1.9220e-03 lr: 1.9220e-03  eta: 6:01:29  time: 0.2488  data_time: 0.0084  memory: 3937  loss: 5.2137  loss_cls: 0.8029  loss_bbox: 2.5394  loss_obj: 1.8713
03/19 18:04:09 - mmengine - INFO - Epoch(train)   [4][ 150/1000]  base_lr: 1.9845e-03 lr: 1.9845e-03  eta: 6:00:56  time: 0.2101  data_time: 0.0085  memory: 3639  loss: 5.2309  loss_cls: 0.8189  loss_bbox: 2.5737  loss_obj: 1.8383
03/19 18:04:20 - mmengine - INFO - Epoch(train)   [4][ 200/1000]  base_lr: 2.0480e-03 lr: 2.0480e-03  eta: 6:00:33  time: 0.2157  data_time: 0.0084  memory: 3937  loss: 5.2693  loss_cls: 0.8194  loss_bbox: 2.5659  loss_obj: 1.8841
03/19 18:04:31 - mmengine - INFO - Epoch(train)   [4][ 250/1000]  base_lr: 2.1125e-03 lr: 2.1125e-03  eta: 6:00:27  time: 0.2267  data_time: 0.0084  memory: 3937  loss: 5.2970  loss_cls: 0.8208  loss_bbox: 2.5586  loss_obj: 1.9177
03/19 18:04:42 - mmengine - INFO - Epoch(train)   [4][ 300/1000]  base_lr: 2.1780e-03 lr: 2.1780e-03  eta: 6:00:04  time: 0.2154  data_time: 0.0085  memory: 3639  loss: 5.1714  loss_cls: 0.8015  loss_bbox: 2.5430  loss_obj: 1.8269
03/19 18:04:53 - mmengine - INFO - Epoch(train)   [4][ 350/1000]  base_lr: 2.2445e-03 lr: 2.2445e-03  eta: 5:59:57  time: 0.2266  data_time: 0.0085  memory: 3937  loss: 5.2769  loss_cls: 0.8140  loss_bbox: 2.5742  loss_obj: 1.8886
03/19 18:05:03 - mmengine - INFO - Epoch(train)   [4][ 400/1000]  base_lr: 2.3120e-03 lr: 2.3120e-03  eta: 5:59:15  time: 0.2014  data_time: 0.0086  memory: 3071  loss: 5.1655  loss_cls: 0.8156  loss_bbox: 2.5577  loss_obj: 1.7921
03/19 18:05:14 - mmengine - INFO - Epoch(train)   [4][ 450/1000]  base_lr: 2.3805e-03 lr: 2.3805e-03  eta: 5:59:05  time: 0.2239  data_time: 0.0085  memory: 3639  loss: 5.2206  loss_cls: 0.8050  loss_bbox: 2.5384  loss_obj: 1.8773
03/19 18:05:25 - mmengine - INFO - Epoch(train)   [4][ 500/1000]  base_lr: 2.4500e-03 lr: 2.4500e-03  eta: 5:58:36  time: 0.2103  data_time: 0.0085  memory: 2820  loss: 5.2496  loss_cls: 0.8218  loss_bbox: 2.5770  loss_obj: 1.8508
03/19 18:05:36 - mmengine - INFO - Epoch(train)   [4][ 550/1000]  base_lr: 2.5205e-03 lr: 2.5205e-03  eta: 5:58:30  time: 0.2269  data_time: 0.0084  memory: 3937  loss: 5.2935  loss_cls: 0.8038  loss_bbox: 2.5483  loss_obj: 1.9413
03/19 18:05:47 - mmengine - INFO - Epoch(train)   [4][ 600/1000]  base_lr: 2.5920e-03 lr: 2.5920e-03  eta: 5:58:02  time: 0.2102  data_time: 0.0086  memory: 3639  loss: 5.3343  loss_cls: 0.8111  loss_bbox: 2.6079  loss_obj: 1.9153
03/19 18:05:58 - mmengine - INFO - Epoch(train)   [4][ 650/1000]  base_lr: 2.6645e-03 lr: 2.6645e-03  eta: 5:57:54  time: 0.2254  data_time: 0.0084  memory: 3639  loss: 5.2202  loss_cls: 0.8143  loss_bbox: 2.5566  loss_obj: 1.8493
03/19 18:06:09 - mmengine - INFO - Epoch(train)   [4][ 700/1000]  base_lr: 2.7380e-03 lr: 2.7380e-03  eta: 5:57:48  time: 0.2266  data_time: 0.0086  memory: 3937  loss: 5.3478  loss_cls: 0.8106  loss_bbox: 2.5954  loss_obj: 1.9417
03/19 18:06:20 - mmengine - INFO - Epoch(train)   [4][ 750/1000]  base_lr: 2.8125e-03 lr: 2.8125e-03  eta: 5:57:23  time: 0.2128  data_time: 0.0085  memory: 3357  loss: 5.4219  loss_cls: 0.8356  loss_bbox: 2.6056  loss_obj: 1.9806
03/19 18:06:30 - mmengine - INFO - Epoch(train)   [4][ 800/1000]  base_lr: 2.8880e-03 lr: 2.8880e-03  eta: 5:56:53  time: 0.2072  data_time: 0.0084  memory: 3639  loss: 5.3043  loss_cls: 0.8190  loss_bbox: 2.6181  loss_obj: 1.8672
03/19 18:06:40 - mmengine - INFO - Epoch(train)   [4][ 850/1000]  base_lr: 2.9645e-03 lr: 2.9645e-03  eta: 5:56:15  time: 0.2012  data_time: 0.0086  memory: 2817  loss: 5.1849  loss_cls: 0.8117  loss_bbox: 2.5704  loss_obj: 1.8028
03/19 18:06:50 - mmengine - INFO - Epoch(train)   [4][ 900/1000]  base_lr: 3.0420e-03 lr: 3.0420e-03  eta: 5:55:32  time: 0.1965  data_time: 0.0085  memory: 2587  loss: 5.1911  loss_cls: 0.8129  loss_bbox: 2.5892  loss_obj: 1.7889
03/19 18:07:01 - mmengine - INFO - Epoch(train)   [4][ 950/1000]  base_lr: 3.1205e-03 lr: 3.1205e-03  eta: 5:55:02  time: 0.2063  data_time: 0.0087  memory: 3639  loss: 5.3382  loss_cls: 0.8165  loss_bbox: 2.6207  loss_obj: 1.9010
03/19 18:07:11 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:07:11 - mmengine - INFO - Epoch(train)   [4][1000/1000]  base_lr: 3.2000e-03 lr: 3.2000e-03  eta: 5:54:30  time: 0.2047  data_time: 0.0084  memory: 2817  loss: 5.3680  loss_cls: 0.8221  loss_bbox: 2.6162  loss_obj: 1.9297
03/19 18:07:11 - mmengine - INFO - Saving checkpoint at 4 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:07:16 - mmengine - INFO - Epoch(val)   [4][ 50/250]    eta: 0:00:12  time: 0.0600  data_time: 0.0072  memory: 527  
03/19 18:07:19 - mmengine - INFO - Epoch(val)   [4][100/250]    eta: 0:00:08  time: 0.0593  data_time: 0.0065  memory: 527  
03/19 18:07:22 - mmengine - INFO - Epoch(val)   [4][150/250]    eta: 0:00:05  time: 0.0595  data_time: 0.0064  memory: 527  
03/19 18:07:25 - mmengine - INFO - Epoch(val)   [4][200/250]    eta: 0:00:02  time: 0.0600  data_time: 0.0065  memory: 527  
03/19 18:07:28 - mmengine - INFO - Epoch(val)   [4][250/250]    eta: 0:00:00  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 18:07:35 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=1.43s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=20.32s).
Accumulating evaluation results...
DONE (t=10.27s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.105
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.270
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.057
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.058
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.266
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.237
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.238
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.184
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.276
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.367
03/19 18:08:09 - mmengine - INFO - bbox_mAP_copypaste: 0.105 0.270 0.057 0.058 0.132 0.266
03/19 18:08:10 - mmengine - INFO - Epoch(val) [4][250/250]    coco/bbox_mAP: 0.1050  coco/bbox_mAP_50: 0.2700  coco/bbox_mAP_75: 0.0570  coco/bbox_mAP_s: 0.0580  coco/bbox_mAP_m: 0.1320  coco/bbox_mAP_l: 0.2660  data_time: 0.0066  time: 0.0594
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:08:20 - mmengine - INFO - Epoch(train)   [5][  50/1000]  base_lr: 3.2805e-03 lr: 3.2805e-03  eta: 5:54:11  time: 0.2144  data_time: 0.0194  memory: 3639  loss: 5.3156  loss_cls: 0.8275  loss_bbox: 2.6108  loss_obj: 1.8773
03/19 18:08:31 - mmengine - INFO - Epoch(train)   [5][ 100/1000]  base_lr: 3.3620e-03 lr: 3.3620e-03  eta: 5:53:50  time: 0.2133  data_time: 0.0086  memory: 3639  loss: 5.4150  loss_cls: 0.8189  loss_bbox: 2.6139  loss_obj: 1.9822
03/19 18:08:43 - mmengine - INFO - Epoch(train)   [5][ 150/1000]  base_lr: 3.4445e-03 lr: 3.4445e-03  eta: 5:53:49  time: 0.2299  data_time: 0.0086  memory: 3937  loss: 5.4523  loss_cls: 0.8351  loss_bbox: 2.6259  loss_obj: 1.9913
03/19 18:08:53 - mmengine - INFO - Epoch(train)   [5][ 200/1000]  base_lr: 3.5280e-03 lr: 3.5280e-03  eta: 5:53:19  time: 0.2052  data_time: 0.0089  memory: 3071  loss: 5.2667  loss_cls: 0.8089  loss_bbox: 2.6148  loss_obj: 1.8430
03/19 18:09:03 - mmengine - INFO - Epoch(train)   [5][ 250/1000]  base_lr: 3.6125e-03 lr: 3.6125e-03  eta: 5:52:46  time: 0.2016  data_time: 0.0087  memory: 2817  loss: 5.3221  loss_cls: 0.8181  loss_bbox: 2.5947  loss_obj: 1.9093
03/19 18:09:13 - mmengine - INFO - Epoch(train)   [5][ 300/1000]  base_lr: 3.6980e-03 lr: 3.6980e-03  eta: 5:52:09  time: 0.1979  data_time: 0.0087  memory: 3639  loss: 5.3104  loss_cls: 0.8246  loss_bbox: 2.6223  loss_obj: 1.8635
03/19 18:09:23 - mmengine - INFO - Epoch(train)   [5][ 350/1000]  base_lr: 3.7845e-03 lr: 3.7845e-03  eta: 5:51:38  time: 0.2025  data_time: 0.0086  memory: 3071  loss: 5.2721  loss_cls: 0.8031  loss_bbox: 2.6063  loss_obj: 1.8628
03/19 18:09:34 - mmengine - INFO - Epoch(train)   [5][ 400/1000]  base_lr: 3.8720e-03 lr: 3.8720e-03  eta: 5:51:18  time: 0.2129  data_time: 0.0086  memory: 3639  loss: 5.3298  loss_cls: 0.8117  loss_bbox: 2.6121  loss_obj: 1.9060
03/19 18:09:45 - mmengine - INFO - Epoch(train)   [5][ 450/1000]  base_lr: 3.9605e-03 lr: 3.9605e-03  eta: 5:51:17  time: 0.2299  data_time: 0.0090  memory: 3937  loss: 5.3616  loss_cls: 0.8184  loss_bbox: 2.5988  loss_obj: 1.9445
03/19 18:09:54 - mmengine - INFO - Epoch(train)   [5][ 500/1000]  base_lr: 4.0500e-03 lr: 4.0500e-03  eta: 5:50:27  time: 0.1834  data_time: 0.0090  memory: 2115  loss: 5.2630  loss_cls: 0.8057  loss_bbox: 2.6262  loss_obj: 1.8311
03/19 18:10:05 - mmengine - INFO - Epoch(train)   [5][ 550/1000]  base_lr: 4.1405e-03 lr: 4.1405e-03  eta: 5:50:16  time: 0.2208  data_time: 0.0085  memory: 3639  loss: 5.3220  loss_cls: 0.8007  loss_bbox: 2.6136  loss_obj: 1.9078
03/19 18:10:15 - mmengine - INFO - Epoch(train)   [5][ 600/1000]  base_lr: 4.2320e-03 lr: 4.2320e-03  eta: 5:49:37  time: 0.1925  data_time: 0.0087  memory: 2817  loss: 5.3155  loss_cls: 0.8225  loss_bbox: 2.6384  loss_obj: 1.8546
03/19 18:10:26 - mmengine - INFO - Epoch(train)   [5][ 650/1000]  base_lr: 4.3245e-03 lr: 4.3245e-03  eta: 5:49:25  time: 0.2194  data_time: 0.0084  memory: 3071  loss: 5.3204  loss_cls: 0.7991  loss_bbox: 2.6095  loss_obj: 1.9119
03/19 18:10:37 - mmengine - INFO - Epoch(train)   [5][ 700/1000]  base_lr: 4.4180e-03 lr: 4.4180e-03  eta: 5:49:22  time: 0.2277  data_time: 0.0085  memory: 3937  loss: 5.3912  loss_cls: 0.8165  loss_bbox: 2.6137  loss_obj: 1.9610
03/19 18:10:48 - mmengine - INFO - Epoch(train)   [5][ 750/1000]  base_lr: 4.5125e-03 lr: 4.5125e-03  eta: 5:49:05  time: 0.2138  data_time: 0.0088  memory: 3937  loss: 5.3447  loss_cls: 0.8158  loss_bbox: 2.6240  loss_obj: 1.9049
03/19 18:10:58 - mmengine - INFO - Epoch(train)   [5][ 800/1000]  base_lr: 4.6080e-03 lr: 4.6080e-03  eta: 5:48:38  time: 0.2041  data_time: 0.0087  memory: 3937  loss: 5.3658  loss_cls: 0.8201  loss_bbox: 2.6416  loss_obj: 1.9041
03/19 18:11:10 - mmengine - INFO - Epoch(train)   [5][ 850/1000]  base_lr: 4.7045e-03 lr: 4.7045e-03  eta: 5:48:39  time: 0.2317  data_time: 0.0085  memory: 3937  loss: 5.4160  loss_cls: 0.8318  loss_bbox: 2.6098  loss_obj: 1.9744
03/19 18:11:20 - mmengine - INFO - Epoch(train)   [5][ 900/1000]  base_lr: 4.8020e-03 lr: 4.8020e-03  eta: 5:48:20  time: 0.2115  data_time: 0.0084  memory: 3357  loss: 5.2728  loss_cls: 0.7981  loss_bbox: 2.5851  loss_obj: 1.8897
03/19 18:11:32 - mmengine - INFO - Epoch(train)   [5][ 950/1000]  base_lr: 4.9005e-03 lr: 4.9005e-03  eta: 5:48:16  time: 0.2270  data_time: 0.0085  memory: 3937  loss: 5.3264  loss_cls: 0.8074  loss_bbox: 2.6226  loss_obj: 1.8963
03/19 18:11:42 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:11:42 - mmengine - INFO - Epoch(train)   [5][1000/1000]  base_lr: 5.0000e-03 lr: 5.0000e-03  eta: 5:47:49  time: 0.2026  data_time: 0.0086  memory: 3357  loss: 5.4067  loss_cls: 0.8247  loss_bbox: 2.6249  loss_obj: 1.9571
03/19 18:11:42 - mmengine - INFO - Saving checkpoint at 5 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:11:47 - mmengine - INFO - Epoch(val)   [5][ 50/250]    eta: 0:00:11  time: 0.0595  data_time: 0.0072  memory: 527  
03/19 18:11:50 - mmengine - INFO - Epoch(val)   [5][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0064  memory: 527  
03/19 18:11:53 - mmengine - INFO - Epoch(val)   [5][150/250]    eta: 0:00:05  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 18:11:56 - mmengine - INFO - Epoch(val)   [5][200/250]    eta: 0:00:02  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 18:11:59 - mmengine - INFO - Epoch(val)   [5][250/250]    eta: 0:00:00  time: 0.0579  data_time: 0.0065  memory: 527  
03/19 18:12:03 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.75s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=18.64s).
Accumulating evaluation results...
DONE (t=5.87s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.111
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.290
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.058
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.064
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.179
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.238
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.239
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.239
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.178
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.286
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.273
03/19 18:12:29 - mmengine - INFO - bbox_mAP_copypaste: 0.111 0.290 0.058 0.064 0.149 0.179
03/19 18:12:30 - mmengine - INFO - Epoch(val) [5][250/250]    coco/bbox_mAP: 0.1110  coco/bbox_mAP_50: 0.2900  coco/bbox_mAP_75: 0.0580  coco/bbox_mAP_s: 0.0640  coco/bbox_mAP_m: 0.1490  coco/bbox_mAP_l: 0.1790  data_time: 0.0066  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:12:41 - mmengine - INFO - Epoch(train)   [6][  50/1000]  base_lr: 4.9999e-03 lr: 4.9999e-03  eta: 5:47:40  time: 0.2222  data_time: 0.0196  memory: 3937  loss: 5.3513  loss_cls: 0.8160  loss_bbox: 2.6291  loss_obj: 1.9062
03/19 18:12:51 - mmengine - INFO - Epoch(train)   [6][ 100/1000]  base_lr: 4.9995e-03 lr: 4.9995e-03  eta: 5:47:17  time: 0.2073  data_time: 0.0087  memory: 3639  loss: 5.3992  loss_cls: 0.8120  loss_bbox: 2.6270  loss_obj: 1.9602
03/19 18:13:02 - mmengine - INFO - Epoch(train)   [6][ 150/1000]  base_lr: 4.9989e-03 lr: 4.9989e-03  eta: 5:46:56  time: 0.2083  data_time: 0.0086  memory: 3357  loss: 5.2731  loss_cls: 0.7996  loss_bbox: 2.6026  loss_obj: 1.8708
03/19 18:13:12 - mmengine - INFO - Epoch(train)   [6][ 200/1000]  base_lr: 4.9980e-03 lr: 4.9980e-03  eta: 5:46:39  time: 0.2132  data_time: 0.0086  memory: 3071  loss: 5.2386  loss_cls: 0.8159  loss_bbox: 2.5755  loss_obj: 1.8471
03/19 18:13:23 - mmengine - INFO - Epoch(train)   [6][ 250/1000]  base_lr: 4.9969e-03 lr: 4.9969e-03  eta: 5:46:30  time: 0.2212  data_time: 0.0087  memory: 3937  loss: 5.3235  loss_cls: 0.8079  loss_bbox: 2.6127  loss_obj: 1.9028
03/19 18:13:35 - mmengine - INFO - Epoch(train)   [6][ 300/1000]  base_lr: 4.9956e-03 lr: 4.9956e-03  eta: 5:46:24  time: 0.2251  data_time: 0.0087  memory: 3639  loss: 5.2881  loss_cls: 0.8061  loss_bbox: 2.5904  loss_obj: 1.8917
03/19 18:13:45 - mmengine - INFO - Epoch(train)   [6][ 350/1000]  base_lr: 4.9940e-03 lr: 4.9940e-03  eta: 5:46:07  time: 0.2127  data_time: 0.0088  memory: 3357  loss: 5.3096  loss_cls: 0.8149  loss_bbox: 2.5936  loss_obj: 1.9010
03/19 18:13:56 - mmengine - INFO - Epoch(train)   [6][ 400/1000]  base_lr: 4.9921e-03 lr: 4.9921e-03  eta: 5:45:58  time: 0.2217  data_time: 0.0088  memory: 3937  loss: 5.2857  loss_cls: 0.7991  loss_bbox: 2.6119  loss_obj: 1.8747
03/19 18:14:07 - mmengine - INFO - Epoch(train)   [6][ 450/1000]  base_lr: 4.9901e-03 lr: 4.9901e-03  eta: 5:45:51  time: 0.2233  data_time: 0.0088  memory: 3639  loss: 5.2172  loss_cls: 0.7881  loss_bbox: 2.5549  loss_obj: 1.8741
03/19 18:14:18 - mmengine - INFO - Epoch(train)   [6][ 500/1000]  base_lr: 4.9877e-03 lr: 4.9877e-03  eta: 5:45:23  time: 0.2007  data_time: 0.0089  memory: 3357  loss: 5.2889  loss_cls: 0.8057  loss_bbox: 2.6240  loss_obj: 1.8592
03/19 18:14:30 - mmengine - INFO - Epoch(train)   [6][ 550/1000]  base_lr: 4.9851e-03 lr: 4.9851e-03  eta: 5:45:35  time: 0.2461  data_time: 0.0089  memory: 3937  loss: 5.3825  loss_cls: 0.8105  loss_bbox: 2.5894  loss_obj: 1.9826
03/19 18:14:41 - mmengine - INFO - Epoch(train)   [6][ 600/1000]  base_lr: 4.9823e-03 lr: 4.9823e-03  eta: 5:45:26  time: 0.2211  data_time: 0.0087  memory: 3639  loss: 5.2354  loss_cls: 0.8045  loss_bbox: 2.5656  loss_obj: 1.8653
03/19 18:14:52 - mmengine - INFO - Epoch(train)   [6][ 650/1000]  base_lr: 4.9792e-03 lr: 4.9792e-03  eta: 5:45:13  time: 0.2171  data_time: 0.0088  memory: 3937  loss: 5.3190  loss_cls: 0.7991  loss_bbox: 2.5943  loss_obj: 1.9256
03/19 18:15:03 - mmengine - INFO - Epoch(train)   [6][ 700/1000]  base_lr: 4.9759e-03 lr: 4.9759e-03  eta: 5:44:58  time: 0.2155  data_time: 0.0086  memory: 3639  loss: 5.3081  loss_cls: 0.8077  loss_bbox: 2.6137  loss_obj: 1.8867
03/19 18:15:13 - mmengine - INFO - Epoch(train)   [6][ 750/1000]  base_lr: 4.9724e-03 lr: 4.9724e-03  eta: 5:44:35  time: 0.2038  data_time: 0.0086  memory: 2817  loss: 5.2327  loss_cls: 0.8093  loss_bbox: 2.6024  loss_obj: 1.8211
03/19 18:15:24 - mmengine - INFO - Epoch(train)   [6][ 800/1000]  base_lr: 4.9686e-03 lr: 4.9686e-03  eta: 5:44:35  time: 0.2329  data_time: 0.0086  memory: 3937  loss: 5.2964  loss_cls: 0.8022  loss_bbox: 2.5865  loss_obj: 1.9076
03/19 18:15:35 - mmengine - INFO - Epoch(train)   [6][ 850/1000]  base_lr: 4.9645e-03 lr: 4.9645e-03  eta: 5:44:18  time: 0.2129  data_time: 0.0086  memory: 3357  loss: 5.2448  loss_cls: 0.7810  loss_bbox: 2.6184  loss_obj: 1.8454
03/19 18:15:46 - mmengine - INFO - Epoch(train)   [6][ 900/1000]  base_lr: 4.9602e-03 lr: 4.9602e-03  eta: 5:44:09  time: 0.2214  data_time: 0.0088  memory: 3639  loss: 5.2855  loss_cls: 0.7928  loss_bbox: 2.5872  loss_obj: 1.9055
03/19 18:15:57 - mmengine - INFO - Epoch(train)   [6][ 950/1000]  base_lr: 4.9557e-03 lr: 4.9557e-03  eta: 5:43:51  time: 0.2109  data_time: 0.0086  memory: 3071  loss: 5.2233  loss_cls: 0.8043  loss_bbox: 2.5776  loss_obj: 1.8414
03/19 18:16:06 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:16:06 - mmengine - INFO - Epoch(train)   [6][1000/1000]  base_lr: 4.9509e-03 lr: 4.9509e-03  eta: 5:43:20  time: 0.1940  data_time: 0.0087  memory: 2587  loss: 5.1189  loss_cls: 0.7885  loss_bbox: 2.5781  loss_obj: 1.7523
03/19 18:16:06 - mmengine - INFO - Saving checkpoint at 6 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:16:12 - mmengine - INFO - Epoch(val)   [6][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0071  memory: 527  
03/19 18:16:15 - mmengine - INFO - Epoch(val)   [6][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 18:16:18 - mmengine - INFO - Epoch(val)   [6][150/250]    eta: 0:00:05  time: 0.0596  data_time: 0.0066  memory: 527  
03/19 18:16:21 - mmengine - INFO - Epoch(val)   [6][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0064  memory: 527  
03/19 18:16:24 - mmengine - INFO - Epoch(val)   [6][250/250]    eta: 0:00:00  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 18:16:26 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.38s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=12.29s).
Accumulating evaluation results...
DONE (t=3.85s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.125
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.323
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.063
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.074
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.166
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.262
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.245
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.245
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.245
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.182
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.289
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.352
03/19 18:16:44 - mmengine - INFO - bbox_mAP_copypaste: 0.125 0.323 0.063 0.074 0.166 0.262
03/19 18:16:44 - mmengine - INFO - Epoch(val) [6][250/250]    coco/bbox_mAP: 0.1250  coco/bbox_mAP_50: 0.3230  coco/bbox_mAP_75: 0.0630  coco/bbox_mAP_s: 0.0740  coco/bbox_mAP_m: 0.1660  coco/bbox_mAP_l: 0.2620  data_time: 0.0066  time: 0.0591
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:16:55 - mmengine - INFO - Epoch(train)   [7][  50/1000]  base_lr: 4.9459e-03 lr: 4.9459e-03  eta: 5:43:10  time: 0.2201  data_time: 0.0193  memory: 3639  loss: 5.1792  loss_cls: 0.7933  loss_bbox: 2.5818  loss_obj: 1.8041
03/19 18:17:06 - mmengine - INFO - Epoch(train)   [7][ 100/1000]  base_lr: 4.9407e-03 lr: 4.9407e-03  eta: 5:43:10  time: 0.2333  data_time: 0.0090  memory: 3937  loss: 5.2669  loss_cls: 0.7981  loss_bbox: 2.5737  loss_obj: 1.8951
03/19 18:17:17 - mmengine - INFO - Epoch(train)   [7][ 150/1000]  base_lr: 4.9352e-03 lr: 4.9352e-03  eta: 5:42:57  time: 0.2165  data_time: 0.0096  memory: 2817  loss: 5.1217  loss_cls: 0.7872  loss_bbox: 2.5580  loss_obj: 1.7765
03/19 18:17:28 - mmengine - INFO - Epoch(train)   [7][ 200/1000]  base_lr: 4.9294e-03 lr: 4.9294e-03  eta: 5:42:49  time: 0.2234  data_time: 0.0087  memory: 3639  loss: 5.1935  loss_cls: 0.7920  loss_bbox: 2.5529  loss_obj: 1.8486
03/19 18:17:39 - mmengine - INFO - Epoch(train)   [7][ 250/1000]  base_lr: 4.9235e-03 lr: 4.9235e-03  eta: 5:42:31  time: 0.2092  data_time: 0.0088  memory: 3639  loss: 5.2071  loss_cls: 0.7908  loss_bbox: 2.6027  loss_obj: 1.8135
03/19 18:17:50 - mmengine - INFO - Epoch(train)   [7][ 300/1000]  base_lr: 4.9172e-03 lr: 4.9172e-03  eta: 5:42:19  time: 0.2183  data_time: 0.0088  memory: 3937  loss: 5.1991  loss_cls: 0.7956  loss_bbox: 2.6071  loss_obj: 1.7964
03/19 18:18:00 - mmengine - INFO - Epoch(train)   [7][ 350/1000]  base_lr: 4.9108e-03 lr: 4.9108e-03  eta: 5:42:01  time: 0.2098  data_time: 0.0088  memory: 3357  loss: 5.1613  loss_cls: 0.7911  loss_bbox: 2.5761  loss_obj: 1.7941
03/19 18:18:11 - mmengine - INFO - Epoch(train)   [7][ 400/1000]  base_lr: 4.9041e-03 lr: 4.9041e-03  eta: 5:41:47  time: 0.2156  data_time: 0.0087  memory: 3357  loss: 5.1907  loss_cls: 0.7961  loss_bbox: 2.5904  loss_obj: 1.8043
03/19 18:18:22 - mmengine - INFO - Epoch(train)   [7][ 450/1000]  base_lr: 4.8972e-03 lr: 4.8972e-03  eta: 5:41:34  time: 0.2163  data_time: 0.0088  memory: 3937  loss: 5.2098  loss_cls: 0.8086  loss_bbox: 2.5732  loss_obj: 1.8280
03/19 18:18:32 - mmengine - INFO - Epoch(train)   [7][ 500/1000]  base_lr: 4.8900e-03 lr: 4.8900e-03  eta: 5:41:11  time: 0.2022  data_time: 0.0088  memory: 3357  loss: 5.2125  loss_cls: 0.7862  loss_bbox: 2.5855  loss_obj: 1.8408
03/19 18:18:42 - mmengine - INFO - Epoch(train)   [7][ 550/1000]  base_lr: 4.8826e-03 lr: 4.8826e-03  eta: 5:40:50  time: 0.2041  data_time: 0.0088  memory: 3357  loss: 5.1940  loss_cls: 0.7906  loss_bbox: 2.5757  loss_obj: 1.8278
03/19 18:18:52 - mmengine - INFO - Epoch(train)   [7][ 600/1000]  base_lr: 4.8750e-03 lr: 4.8750e-03  eta: 5:40:28  time: 0.2029  data_time: 0.0086  memory: 3071  loss: 5.2044  loss_cls: 0.7968  loss_bbox: 2.5919  loss_obj: 1.8157
03/19 18:19:02 - mmengine - INFO - Epoch(train)   [7][ 650/1000]  base_lr: 4.8671e-03 lr: 4.8671e-03  eta: 5:40:01  time: 0.1957  data_time: 0.0088  memory: 3071  loss: 5.1502  loss_cls: 0.7854  loss_bbox: 2.6060  loss_obj: 1.7588
03/19 18:19:12 - mmengine - INFO - Epoch(train)   [7][ 700/1000]  base_lr: 4.8590e-03 lr: 4.8590e-03  eta: 5:39:37  time: 0.2003  data_time: 0.0089  memory: 2817  loss: 5.1094  loss_cls: 0.7998  loss_bbox: 2.5714  loss_obj: 1.7382
03/19 18:19:22 - mmengine - INFO - Epoch(train)   [7][ 750/1000]  base_lr: 4.8507e-03 lr: 4.8507e-03  eta: 5:39:16  time: 0.2034  data_time: 0.0087  memory: 3639  loss: 5.0811  loss_cls: 0.7770  loss_bbox: 2.5471  loss_obj: 1.7569
03/19 18:19:34 - mmengine - INFO - Epoch(train)   [7][ 800/1000]  base_lr: 4.8422e-03 lr: 4.8422e-03  eta: 5:39:08  time: 0.2233  data_time: 0.0086  memory: 3357  loss: 5.1524  loss_cls: 0.7927  loss_bbox: 2.5513  loss_obj: 1.8084
03/19 18:19:44 - mmengine - INFO - Epoch(train)   [7][ 850/1000]  base_lr: 4.8334e-03 lr: 4.8334e-03  eta: 5:38:56  time: 0.2165  data_time: 0.0086  memory: 3357  loss: 5.1413  loss_cls: 0.7826  loss_bbox: 2.5630  loss_obj: 1.7957
03/19 18:19:55 - mmengine - INFO - Epoch(train)   [7][ 900/1000]  base_lr: 4.8244e-03 lr: 4.8244e-03  eta: 5:38:36  time: 0.2047  data_time: 0.0087  memory: 3639  loss: 5.1123  loss_cls: 0.7797  loss_bbox: 2.5763  loss_obj: 1.7564
03/19 18:20:05 - mmengine - INFO - Epoch(train)   [7][ 950/1000]  base_lr: 4.8151e-03 lr: 4.8151e-03  eta: 5:38:16  time: 0.2054  data_time: 0.0089  memory: 3357  loss: 5.0939  loss_cls: 0.7841  loss_bbox: 2.5672  loss_obj: 1.7426
03/19 18:20:16 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:20:16 - mmengine - INFO - Epoch(train)   [7][1000/1000]  base_lr: 4.8057e-03 lr: 4.8057e-03  eta: 5:38:01  time: 0.2120  data_time: 0.0087  memory: 3639  loss: 5.2247  loss_cls: 0.7910  loss_bbox: 2.5771  loss_obj: 1.8567
03/19 18:20:16 - mmengine - INFO - Saving checkpoint at 7 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:20:21 - mmengine - INFO - Epoch(val)   [7][ 50/250]    eta: 0:00:12  time: 0.0601  data_time: 0.0071  memory: 527  
03/19 18:20:24 - mmengine - INFO - Epoch(val)   [7][100/250]    eta: 0:00:08  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 18:20:27 - mmengine - INFO - Epoch(val)   [7][150/250]    eta: 0:00:05  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 18:20:30 - mmengine - INFO - Epoch(val)   [7][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 18:20:33 - mmengine - INFO - Epoch(val)   [7][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0065  memory: 527  
03/19 18:20:36 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.64s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=13.63s).
Accumulating evaluation results...
DONE (t=4.68s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.343
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.091
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.078
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.175
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.277
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.262
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.262
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.262
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.201
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.352
03/19 18:20:56 - mmengine - INFO - bbox_mAP_copypaste: 0.138 0.343 0.091 0.078 0.175 0.277
03/19 18:20:56 - mmengine - INFO - Epoch(val) [7][250/250]    coco/bbox_mAP: 0.1380  coco/bbox_mAP_50: 0.3430  coco/bbox_mAP_75: 0.0910  coco/bbox_mAP_s: 0.0780  coco/bbox_mAP_m: 0.1750  coco/bbox_mAP_l: 0.2770  data_time: 0.0066  time: 0.0590
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:21:07 - mmengine - INFO - Epoch(train)   [8][  50/1000]  base_lr: 4.7960e-03 lr: 4.7960e-03  eta: 5:37:56  time: 0.2269  data_time: 0.0198  memory: 3071  loss: 5.0627  loss_cls: 0.7755  loss_bbox: 2.5441  loss_obj: 1.7431
03/19 18:21:18 - mmengine - INFO - Epoch(train)   [8][ 100/1000]  base_lr: 4.7861e-03 lr: 4.7861e-03  eta: 5:37:48  time: 0.2228  data_time: 0.0086  memory: 3639  loss: 5.0461  loss_cls: 0.7729  loss_bbox: 2.5390  loss_obj: 1.7342
03/19 18:21:29 - mmengine - INFO - Epoch(train)   [8][ 150/1000]  base_lr: 4.7759e-03 lr: 4.7759e-03  eta: 5:37:34  time: 0.2132  data_time: 0.0086  memory: 3639  loss: 5.0692  loss_cls: 0.7740  loss_bbox: 2.5399  loss_obj: 1.7553
03/19 18:21:40 - mmengine - INFO - Epoch(train)   [8][ 200/1000]  base_lr: 4.7656e-03 lr: 4.7656e-03  eta: 5:37:20  time: 0.2127  data_time: 0.0086  memory: 3071  loss: 5.0222  loss_cls: 0.7694  loss_bbox: 2.5270  loss_obj: 1.7258
03/19 18:21:51 - mmengine - INFO - Epoch(train)   [8][ 250/1000]  base_lr: 4.7550e-03 lr: 4.7550e-03  eta: 5:37:20  time: 0.2354  data_time: 0.0085  memory: 3639  loss: 5.0551  loss_cls: 0.7608  loss_bbox: 2.5188  loss_obj: 1.7755
03/19 18:22:03 - mmengine - INFO - Epoch(train)   [8][ 300/1000]  base_lr: 4.7442e-03 lr: 4.7442e-03  eta: 5:37:21  time: 0.2374  data_time: 0.0086  memory: 3937  loss: 5.0933  loss_cls: 0.7654  loss_bbox: 2.5396  loss_obj: 1.7884
03/19 18:22:15 - mmengine - INFO - Epoch(train)   [8][ 350/1000]  base_lr: 4.7332e-03 lr: 4.7332e-03  eta: 5:37:15  time: 0.2262  data_time: 0.0086  memory: 3937  loss: 5.1078  loss_cls: 0.7732  loss_bbox: 2.5733  loss_obj: 1.7613
03/19 18:22:26 - mmengine - INFO - Epoch(train)   [8][ 400/1000]  base_lr: 4.7219e-03 lr: 4.7219e-03  eta: 5:37:12  time: 0.2299  data_time: 0.0086  memory: 3937  loss: 5.0714  loss_cls: 0.7636  loss_bbox: 2.5206  loss_obj: 1.7872
03/19 18:22:37 - mmengine - INFO - Epoch(train)   [8][ 450/1000]  base_lr: 4.7105e-03 lr: 4.7105e-03  eta: 5:36:57  time: 0.2121  data_time: 0.0087  memory: 2817  loss: 5.0297  loss_cls: 0.7708  loss_bbox: 2.5368  loss_obj: 1.7220
03/19 18:22:48 - mmengine - INFO - Epoch(train)   [8][ 500/1000]  base_lr: 4.6988e-03 lr: 4.6988e-03  eta: 5:36:49  time: 0.2236  data_time: 0.0088  memory: 3357  loss: 5.0747  loss_cls: 0.7722  loss_bbox: 2.5164  loss_obj: 1.7862
03/19 18:22:58 - mmengine - INFO - Epoch(train)   [8][ 550/1000]  base_lr: 4.6869e-03 lr: 4.6869e-03  eta: 5:36:25  time: 0.1980  data_time: 0.0088  memory: 3357  loss: 5.0194  loss_cls: 0.7622  loss_bbox: 2.5591  loss_obj: 1.6982
03/19 18:23:07 - mmengine - INFO - Epoch(train)   [8][ 600/1000]  base_lr: 4.6748e-03 lr: 4.6748e-03  eta: 5:35:55  time: 0.1863  data_time: 0.0088  memory: 2337  loss: 5.0592  loss_cls: 0.7756  loss_bbox: 2.5525  loss_obj: 1.7311
03/19 18:23:18 - mmengine - INFO - Epoch(train)   [8][ 650/1000]  base_lr: 4.6625e-03 lr: 4.6625e-03  eta: 5:35:43  time: 0.2161  data_time: 0.0087  memory: 3639  loss: 4.9702  loss_cls: 0.7554  loss_bbox: 2.5255  loss_obj: 1.6893
03/19 18:23:28 - mmengine - INFO - Epoch(train)   [8][ 700/1000]  base_lr: 4.6500e-03 lr: 4.6500e-03  eta: 5:35:18  time: 0.1942  data_time: 0.0088  memory: 2337  loss: 5.0062  loss_cls: 0.7795  loss_bbox: 2.5493  loss_obj: 1.6775
03/19 18:23:39 - mmengine - INFO - Epoch(train)   [8][ 750/1000]  base_lr: 4.6373e-03 lr: 4.6373e-03  eta: 5:35:06  time: 0.2167  data_time: 0.0088  memory: 3639  loss: 5.0732  loss_cls: 0.7628  loss_bbox: 2.5516  loss_obj: 1.7587
03/19 18:23:50 - mmengine - INFO - Epoch(train)   [8][ 800/1000]  base_lr: 4.6243e-03 lr: 4.6243e-03  eta: 5:34:59  time: 0.2244  data_time: 0.0088  memory: 3937  loss: 5.1162  loss_cls: 0.7819  loss_bbox: 2.5470  loss_obj: 1.7873
03/19 18:24:02 - mmengine - INFO - Epoch(train)   [8][ 850/1000]  base_lr: 4.6112e-03 lr: 4.6112e-03  eta: 5:35:06  time: 0.2484  data_time: 0.0086  memory: 3937  loss: 5.0404  loss_cls: 0.7603  loss_bbox: 2.5320  loss_obj: 1.7482
03/19 18:24:13 - mmengine - INFO - Epoch(train)   [8][ 900/1000]  base_lr: 4.5979e-03 lr: 4.5979e-03  eta: 5:34:56  time: 0.2197  data_time: 0.0086  memory: 3357  loss: 4.9365  loss_cls: 0.7615  loss_bbox: 2.4988  loss_obj: 1.6762
03/19 18:24:24 - mmengine - INFO - Epoch(train)   [8][ 950/1000]  base_lr: 4.5843e-03 lr: 4.5843e-03  eta: 5:34:39  time: 0.2083  data_time: 0.0086  memory: 3071  loss: 4.9455  loss_cls: 0.7569  loss_bbox: 2.5127  loss_obj: 1.6759
03/19 18:24:35 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:24:35 - mmengine - INFO - Epoch(train)   [8][1000/1000]  base_lr: 4.5706e-03 lr: 4.5706e-03  eta: 5:34:29  time: 0.2188  data_time: 0.0085  memory: 3937  loss: 5.0375  loss_cls: 0.7717  loss_bbox: 2.5336  loss_obj: 1.7321
03/19 18:24:35 - mmengine - INFO - Saving checkpoint at 8 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:24:40 - mmengine - INFO - Epoch(val)   [8][ 50/250]    eta: 0:00:12  time: 0.0603  data_time: 0.0072  memory: 527  
03/19 18:24:43 - mmengine - INFO - Epoch(val)   [8][100/250]    eta: 0:00:08  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 18:24:46 - mmengine - INFO - Epoch(val)   [8][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 18:24:49 - mmengine - INFO - Epoch(val)   [8][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 18:24:52 - mmengine - INFO - Epoch(val)   [8][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0065  memory: 527  
03/19 18:24:54 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.38s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=12.04s).
Accumulating evaluation results...
DONE (t=3.71s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.154
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.380
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.096
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.089
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.192
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.362
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.203
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.431
03/19 18:25:11 - mmengine - INFO - bbox_mAP_copypaste: 0.154 0.380 0.096 0.089 0.192 0.362
03/19 18:25:11 - mmengine - INFO - Epoch(val) [8][250/250]    coco/bbox_mAP: 0.1540  coco/bbox_mAP_50: 0.3800  coco/bbox_mAP_75: 0.0960  coco/bbox_mAP_s: 0.0890  coco/bbox_mAP_m: 0.1920  coco/bbox_mAP_l: 0.3620  data_time: 0.0066  time: 0.0588
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:25:23 - mmengine - INFO - Epoch(train)   [9][  50/1000]  base_lr: 4.5566e-03 lr: 4.5566e-03  eta: 5:34:23  time: 0.2274  data_time: 0.0190  memory: 3639  loss: 4.9667  loss_cls: 0.7635  loss_bbox: 2.5003  loss_obj: 1.7030
03/19 18:25:34 - mmengine - INFO - Epoch(train)   [9][ 100/1000]  base_lr: 4.5425e-03 lr: 4.5425e-03  eta: 5:34:15  time: 0.2224  data_time: 0.0086  memory: 3639  loss: 5.0153  loss_cls: 0.7799  loss_bbox: 2.5196  loss_obj: 1.7158
03/19 18:25:46 - mmengine - INFO - Epoch(train)   [9][ 150/1000]  base_lr: 4.5281e-03 lr: 4.5281e-03  eta: 5:34:18  time: 0.2437  data_time: 0.0086  memory: 3937  loss: 5.0452  loss_cls: 0.7608  loss_bbox: 2.5586  loss_obj: 1.7258
03/19 18:25:58 - mmengine - INFO - Epoch(train)   [9][ 200/1000]  base_lr: 4.5136e-03 lr: 4.5136e-03  eta: 5:34:19  time: 0.2391  data_time: 0.0085  memory: 3937  loss: 5.0227  loss_cls: 0.7606  loss_bbox: 2.5070  loss_obj: 1.7551
03/19 18:26:09 - mmengine - INFO - Epoch(train)   [9][ 250/1000]  base_lr: 4.4989e-03 lr: 4.4989e-03  eta: 5:34:09  time: 0.2201  data_time: 0.0085  memory: 3639  loss: 4.9528  loss_cls: 0.7572  loss_bbox: 2.4628  loss_obj: 1.7328
03/19 18:26:20 - mmengine - INFO - Epoch(train)   [9][ 300/1000]  base_lr: 4.4840e-03 lr: 4.4840e-03  eta: 5:33:56  time: 0.2159  data_time: 0.0086  memory: 3937  loss: 4.9677  loss_cls: 0.7599  loss_bbox: 2.5025  loss_obj: 1.7052
03/19 18:26:29 - mmengine - INFO - Epoch(train)   [9][ 350/1000]  base_lr: 4.4688e-03 lr: 4.4688e-03  eta: 5:33:27  time: 0.1852  data_time: 0.0087  memory: 2115  loss: 5.0015  loss_cls: 0.7859  loss_bbox: 2.5522  loss_obj: 1.6634
03/19 18:26:40 - mmengine - INFO - Epoch(train)   [9][ 400/1000]  base_lr: 4.4535e-03 lr: 4.4535e-03  eta: 5:33:16  time: 0.2174  data_time: 0.0087  memory: 3639  loss: 4.9541  loss_cls: 0.7530  loss_bbox: 2.5039  loss_obj: 1.6972
03/19 18:26:51 - mmengine - INFO - Epoch(train)   [9][ 450/1000]  base_lr: 4.4381e-03 lr: 4.4381e-03  eta: 5:33:05  time: 0.2181  data_time: 0.0086  memory: 3937  loss: 5.0032  loss_cls: 0.7592  loss_bbox: 2.5082  loss_obj: 1.7359
03/19 18:27:01 - mmengine - INFO - Epoch(train)   [9][ 500/1000]  base_lr: 4.4224e-03 lr: 4.4224e-03  eta: 5:32:44  time: 0.2002  data_time: 0.0089  memory: 3357  loss: 4.9675  loss_cls: 0.7652  loss_bbox: 2.5327  loss_obj: 1.6695
03/19 18:27:11 - mmengine - INFO - Epoch(train)   [9][ 550/1000]  base_lr: 4.4065e-03 lr: 4.4065e-03  eta: 5:32:23  time: 0.1993  data_time: 0.0089  memory: 3357  loss: 5.0333  loss_cls: 0.7764  loss_bbox: 2.5338  loss_obj: 1.7232
03/19 18:27:23 - mmengine - INFO - Epoch(train)   [9][ 600/1000]  base_lr: 4.3905e-03 lr: 4.3905e-03  eta: 5:32:20  time: 0.2334  data_time: 0.0088  memory: 3937  loss: 4.9504  loss_cls: 0.7624  loss_bbox: 2.4959  loss_obj: 1.6921
03/19 18:27:33 - mmengine - INFO - Epoch(train)   [9][ 650/1000]  base_lr: 4.3743e-03 lr: 4.3743e-03  eta: 5:31:59  time: 0.1992  data_time: 0.0090  memory: 3937  loss: 4.9866  loss_cls: 0.7668  loss_bbox: 2.5540  loss_obj: 1.6658
03/19 18:27:44 - mmengine - INFO - Epoch(train)   [9][ 700/1000]  base_lr: 4.3579e-03 lr: 4.3579e-03  eta: 5:31:49  time: 0.2197  data_time: 0.0086  memory: 3071  loss: 4.9579  loss_cls: 0.7468  loss_bbox: 2.5037  loss_obj: 1.7074
03/19 18:27:55 - mmengine - INFO - Epoch(train)   [9][ 750/1000]  base_lr: 4.3413e-03 lr: 4.3413e-03  eta: 5:31:48  time: 0.2363  data_time: 0.0088  memory: 3937  loss: 4.9146  loss_cls: 0.7581  loss_bbox: 2.4951  loss_obj: 1.6614
03/19 18:28:06 - mmengine - INFO - Epoch(train)   [9][ 800/1000]  base_lr: 4.3246e-03 lr: 4.3246e-03  eta: 5:31:36  time: 0.2168  data_time: 0.0086  memory: 3357  loss: 5.0070  loss_cls: 0.7601  loss_bbox: 2.5228  loss_obj: 1.7241
03/19 18:28:18 - mmengine - INFO - Epoch(train)   [9][ 850/1000]  base_lr: 4.3077e-03 lr: 4.3077e-03  eta: 5:31:33  time: 0.2325  data_time: 0.0087  memory: 3937  loss: 5.0018  loss_cls: 0.7658  loss_bbox: 2.5119  loss_obj: 1.7242
03/19 18:28:29 - mmengine - INFO - Epoch(train)   [9][ 900/1000]  base_lr: 4.2906e-03 lr: 4.2906e-03  eta: 5:31:27  time: 0.2286  data_time: 0.0091  memory: 3937  loss: 4.9585  loss_cls: 0.7564  loss_bbox: 2.5107  loss_obj: 1.6915
03/19 18:28:40 - mmengine - INFO - Epoch(train)   [9][ 950/1000]  base_lr: 4.2733e-03 lr: 4.2733e-03  eta: 5:31:19  time: 0.2228  data_time: 0.0087  memory: 3937  loss: 5.0320  loss_cls: 0.7692  loss_bbox: 2.5376  loss_obj: 1.7253
03/19 18:28:52 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:28:52 - mmengine - INFO - Epoch(train)   [9][1000/1000]  base_lr: 4.2559e-03 lr: 4.2559e-03  eta: 5:31:18  time: 0.2379  data_time: 0.0085  memory: 3937  loss: 4.8850  loss_cls: 0.7400  loss_bbox: 2.4728  loss_obj: 1.6723
03/19 18:28:52 - mmengine - INFO - Saving checkpoint at 9 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:28:58 - mmengine - INFO - Epoch(val)   [9][ 50/250]    eta: 0:00:12  time: 0.0606  data_time: 0.0074  memory: 527  
03/19 18:29:01 - mmengine - INFO - Epoch(val)   [9][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 18:29:04 - mmengine - INFO - Epoch(val)   [9][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 18:29:06 - mmengine - INFO - Epoch(val)   [9][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:29:09 - mmengine - INFO - Epoch(val)   [9][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0064  memory: 527  
03/19 18:29:12 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.59s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=12.68s).
Accumulating evaluation results...
DONE (t=3.95s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.160
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.393
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.103
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.093
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.219
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.281
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.281
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.281
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.212
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.359
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.405
03/19 18:29:30 - mmengine - INFO - bbox_mAP_copypaste: 0.160 0.393 0.103 0.093 0.219 0.334
03/19 18:29:30 - mmengine - INFO - Epoch(val) [9][250/250]    coco/bbox_mAP: 0.1600  coco/bbox_mAP_50: 0.3930  coco/bbox_mAP_75: 0.1030  coco/bbox_mAP_s: 0.0930  coco/bbox_mAP_m: 0.2190  coco/bbox_mAP_l: 0.3340  data_time: 0.0067  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:29:43 - mmengine - INFO - Epoch(train)  [10][  50/1000]  base_lr: 4.2383e-03 lr: 4.2383e-03  eta: 5:31:21  time: 0.2474  data_time: 0.0192  memory: 3937  loss: 5.0600  loss_cls: 0.7765  loss_bbox: 2.5407  loss_obj: 1.7427
03/19 18:29:54 - mmengine - INFO - Epoch(train)  [10][ 100/1000]  base_lr: 4.2206e-03 lr: 4.2206e-03  eta: 5:31:13  time: 0.2248  data_time: 0.0086  memory: 3639  loss: 5.0052  loss_cls: 0.7688  loss_bbox: 2.5291  loss_obj: 1.7074
03/19 18:30:05 - mmengine - INFO - Epoch(train)  [10][ 150/1000]  base_lr: 4.2026e-03 lr: 4.2026e-03  eta: 5:31:02  time: 0.2185  data_time: 0.0087  memory: 3937  loss: 5.0332  loss_cls: 0.7745  loss_bbox: 2.5416  loss_obj: 1.7172
03/19 18:30:16 - mmengine - INFO - Epoch(train)  [10][ 200/1000]  base_lr: 4.1846e-03 lr: 4.1846e-03  eta: 5:30:50  time: 0.2166  data_time: 0.0086  memory: 3071  loss: 4.9744  loss_cls: 0.7738  loss_bbox: 2.5158  loss_obj: 1.6848
03/19 18:30:25 - mmengine - INFO - Epoch(train)  [10][ 250/1000]  base_lr: 4.1663e-03 lr: 4.1663e-03  eta: 5:30:25  time: 0.1888  data_time: 0.0087  memory: 2337  loss: 4.8634  loss_cls: 0.7548  loss_bbox: 2.5201  loss_obj: 1.5884
03/19 18:30:36 - mmengine - INFO - Epoch(train)  [10][ 300/1000]  base_lr: 4.1480e-03 lr: 4.1480e-03  eta: 5:30:16  time: 0.2223  data_time: 0.0087  memory: 3357  loss: 4.9188  loss_cls: 0.7604  loss_bbox: 2.4989  loss_obj: 1.6596
03/19 18:30:47 - mmengine - INFO - Epoch(train)  [10][ 350/1000]  base_lr: 4.1294e-03 lr: 4.1294e-03  eta: 5:30:03  time: 0.2144  data_time: 0.0087  memory: 3357  loss: 4.8823  loss_cls: 0.7496  loss_bbox: 2.4992  loss_obj: 1.6335
03/19 18:30:59 - mmengine - INFO - Epoch(train)  [10][ 400/1000]  base_lr: 4.1107e-03 lr: 4.1107e-03  eta: 5:30:06  time: 0.2471  data_time: 0.0087  memory: 3937  loss: 4.9656  loss_cls: 0.7382  loss_bbox: 2.5232  loss_obj: 1.7042
03/19 18:31:10 - mmengine - INFO - Epoch(train)  [10][ 450/1000]  base_lr: 4.0919e-03 lr: 4.0919e-03  eta: 5:29:54  time: 0.2167  data_time: 0.0087  memory: 3639  loss: 4.9889  loss_cls: 0.7589  loss_bbox: 2.5275  loss_obj: 1.7025
03/19 18:31:20 - mmengine - INFO - Epoch(train)  [10][ 500/1000]  base_lr: 4.0729e-03 lr: 4.0729e-03  eta: 5:29:37  time: 0.2050  data_time: 0.0086  memory: 3357  loss: 4.8154  loss_cls: 0.7472  loss_bbox: 2.4911  loss_obj: 1.5771
03/19 18:31:31 - mmengine - INFO - Epoch(train)  [10][ 550/1000]  base_lr: 4.0538e-03 lr: 4.0538e-03  eta: 5:29:20  time: 0.2064  data_time: 0.0087  memory: 3639  loss: 4.8522  loss_cls: 0.7474  loss_bbox: 2.4806  loss_obj: 1.6241
03/19 18:31:42 - mmengine - INFO - Epoch(train)  [10][ 600/1000]  base_lr: 4.0345e-03 lr: 4.0345e-03  eta: 5:29:12  time: 0.2241  data_time: 0.0086  memory: 3937  loss: 4.9551  loss_cls: 0.7564  loss_bbox: 2.4959  loss_obj: 1.7027
03/19 18:31:52 - mmengine - INFO - Epoch(train)  [10][ 650/1000]  base_lr: 4.0151e-03 lr: 4.0151e-03  eta: 5:28:56  time: 0.2079  data_time: 0.0087  memory: 3639  loss: 4.9620  loss_cls: 0.7607  loss_bbox: 2.5148  loss_obj: 1.6865
03/19 18:32:03 - mmengine - INFO - Epoch(train)  [10][ 700/1000]  base_lr: 3.9955e-03 lr: 3.9955e-03  eta: 5:28:43  time: 0.2153  data_time: 0.0086  memory: 2817  loss: 4.9722  loss_cls: 0.7598  loss_bbox: 2.5127  loss_obj: 1.6997
03/19 18:32:14 - mmengine - INFO - Epoch(train)  [10][ 750/1000]  base_lr: 3.9758e-03 lr: 3.9758e-03  eta: 5:28:27  time: 0.2075  data_time: 0.0087  memory: 3639  loss: 4.8982  loss_cls: 0.7698  loss_bbox: 2.5062  loss_obj: 1.6221
03/19 18:32:25 - mmengine - INFO - Epoch(train)  [10][ 800/1000]  base_lr: 3.9560e-03 lr: 3.9560e-03  eta: 5:28:18  time: 0.2225  data_time: 0.0088  memory: 3357  loss: 4.8706  loss_cls: 0.7454  loss_bbox: 2.4854  loss_obj: 1.6398
03/19 18:32:36 - mmengine - INFO - Epoch(train)  [10][ 850/1000]  base_lr: 3.9361e-03 lr: 3.9361e-03  eta: 5:28:11  time: 0.2269  data_time: 0.0088  memory: 3937  loss: 4.9153  loss_cls: 0.7525  loss_bbox: 2.4877  loss_obj: 1.6751
03/19 18:32:47 - mmengine - INFO - Epoch(train)  [10][ 900/1000]  base_lr: 3.9160e-03 lr: 3.9160e-03  eta: 5:27:57  time: 0.2106  data_time: 0.0089  memory: 3937  loss: 4.8919  loss_cls: 0.7556  loss_bbox: 2.5118  loss_obj: 1.6245
03/19 18:32:57 - mmengine - INFO - Epoch(train)  [10][ 950/1000]  base_lr: 3.8957e-03 lr: 3.8957e-03  eta: 5:27:43  time: 0.2123  data_time: 0.0089  memory: 3639  loss: 4.8297  loss_cls: 0.7490  loss_bbox: 2.4836  loss_obj: 1.5971
03/19 18:33:10 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:33:10 - mmengine - INFO - Epoch(train)  [10][1000/1000]  base_lr: 3.8754e-03 lr: 3.8754e-03  eta: 5:27:46  time: 0.2498  data_time: 0.0086  memory: 3937  loss: 4.8838  loss_cls: 0.7472  loss_bbox: 2.4570  loss_obj: 1.6795
03/19 18:33:10 - mmengine - INFO - Saving checkpoint at 10 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:33:15 - mmengine - INFO - Epoch(val)  [10][ 50/250]    eta: 0:00:12  time: 0.0603  data_time: 0.0073  memory: 527  
03/19 18:33:18 - mmengine - INFO - Epoch(val)  [10][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 18:33:21 - mmengine - INFO - Epoch(val)  [10][150/250]    eta: 0:00:05  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 18:33:24 - mmengine - INFO - Epoch(val)  [10][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 18:33:27 - mmengine - INFO - Epoch(val)  [10][250/250]    eta: 0:00:00  time: 0.0569  data_time: 0.0065  memory: 527  
03/19 18:33:29 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.30s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.41s).
Accumulating evaluation results...
DONE (t=2.78s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.168
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.407
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.100
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.207
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.278
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.278
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.278
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.210
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.330
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.408
03/19 18:33:42 - mmengine - INFO - bbox_mAP_copypaste: 0.168 0.407 0.113 0.100 0.207 0.334
03/19 18:33:42 - mmengine - INFO - Epoch(val) [10][250/250]    coco/bbox_mAP: 0.1680  coco/bbox_mAP_50: 0.4070  coco/bbox_mAP_75: 0.1130  coco/bbox_mAP_s: 0.1000  coco/bbox_mAP_m: 0.2070  coco/bbox_mAP_l: 0.3340  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:33:54 - mmengine - INFO - Epoch(train)  [11][  50/1000]  base_lr: 3.8549e-03 lr: 3.8549e-03  eta: 5:27:48  time: 0.2477  data_time: 0.0190  memory: 3639  loss: 4.8774  loss_cls: 0.7491  loss_bbox: 2.4819  loss_obj: 1.6464
03/19 18:34:05 - mmengine - INFO - Epoch(train)  [11][ 100/1000]  base_lr: 3.8344e-03 lr: 3.8344e-03  eta: 5:27:34  time: 0.2118  data_time: 0.0087  memory: 3357  loss: 4.8540  loss_cls: 0.7598  loss_bbox: 2.4643  loss_obj: 1.6299
03/19 18:34:15 - mmengine - INFO - Epoch(train)  [11][ 150/1000]  base_lr: 3.8137e-03 lr: 3.8137e-03  eta: 5:27:22  time: 0.2155  data_time: 0.0087  memory: 3639  loss: 4.8945  loss_cls: 0.7591  loss_bbox: 2.5122  loss_obj: 1.6232
03/19 18:34:26 - mmengine - INFO - Epoch(train)  [11][ 200/1000]  base_lr: 3.7928e-03 lr: 3.7928e-03  eta: 5:27:11  time: 0.2187  data_time: 0.0086  memory: 3639  loss: 4.9056  loss_cls: 0.7618  loss_bbox: 2.5063  loss_obj: 1.6375
03/19 18:34:37 - mmengine - INFO - Epoch(train)  [11][ 250/1000]  base_lr: 3.7719e-03 lr: 3.7719e-03  eta: 5:26:54  time: 0.2037  data_time: 0.0086  memory: 2817  loss: 4.8558  loss_cls: 0.7568  loss_bbox: 2.4972  loss_obj: 1.6018
03/19 18:34:47 - mmengine - INFO - Epoch(train)  [11][ 300/1000]  base_lr: 3.7509e-03 lr: 3.7509e-03  eta: 5:26:36  time: 0.2033  data_time: 0.0089  memory: 3937  loss: 4.8397  loss_cls: 0.7440  loss_bbox: 2.4773  loss_obj: 1.6185
03/19 18:34:58 - mmengine - INFO - Epoch(train)  [11][ 350/1000]  base_lr: 3.7297e-03 lr: 3.7297e-03  eta: 5:26:24  time: 0.2160  data_time: 0.0088  memory: 3357  loss: 4.8870  loss_cls: 0.7518  loss_bbox: 2.4807  loss_obj: 1.6546
03/19 18:35:09 - mmengine - INFO - Epoch(train)  [11][ 400/1000]  base_lr: 3.7084e-03 lr: 3.7084e-03  eta: 5:26:19  time: 0.2321  data_time: 0.0088  memory: 3937  loss: 4.8697  loss_cls: 0.7469  loss_bbox: 2.4830  loss_obj: 1.6399
03/19 18:35:21 - mmengine - INFO - Epoch(train)  [11][ 450/1000]  base_lr: 3.6871e-03 lr: 3.6871e-03  eta: 5:26:19  time: 0.2443  data_time: 0.0087  memory: 3639  loss: 4.8660  loss_cls: 0.7476  loss_bbox: 2.4795  loss_obj: 1.6389
03/19 18:35:31 - mmengine - INFO - Epoch(train)  [11][ 500/1000]  base_lr: 3.6656e-03 lr: 3.6656e-03  eta: 5:25:53  time: 0.1836  data_time: 0.0089  memory: 3071  loss: 5.0036  loss_cls: 0.7739  loss_bbox: 2.5501  loss_obj: 1.6797
03/19 18:35:43 - mmengine - INFO - Epoch(train)  [11][ 550/1000]  base_lr: 3.6440e-03 lr: 3.6440e-03  eta: 5:25:53  time: 0.2446  data_time: 0.0088  memory: 3937  loss: 4.8994  loss_cls: 0.7434  loss_bbox: 2.4888  loss_obj: 1.6672
03/19 18:35:54 - mmengine - INFO - Epoch(train)  [11][ 600/1000]  base_lr: 3.6223e-03 lr: 3.6223e-03  eta: 5:25:40  time: 0.2127  data_time: 0.0088  memory: 3071  loss: 4.9175  loss_cls: 0.7495  loss_bbox: 2.4969  loss_obj: 1.6711
03/19 18:36:04 - mmengine - INFO - Epoch(train)  [11][ 650/1000]  base_lr: 3.6006e-03 lr: 3.6006e-03  eta: 5:25:28  time: 0.2149  data_time: 0.0089  memory: 3937  loss: 4.8565  loss_cls: 0.7498  loss_bbox: 2.4941  loss_obj: 1.6127
03/19 18:36:16 - mmengine - INFO - Epoch(train)  [11][ 700/1000]  base_lr: 3.5787e-03 lr: 3.5787e-03  eta: 5:25:22  time: 0.2319  data_time: 0.0086  memory: 3357  loss: 4.8761  loss_cls: 0.7552  loss_bbox: 2.4654  loss_obj: 1.6555
03/19 18:36:28 - mmengine - INFO - Epoch(train)  [11][ 750/1000]  base_lr: 3.5568e-03 lr: 3.5568e-03  eta: 5:25:20  time: 0.2384  data_time: 0.0087  memory: 3937  loss: 4.7623  loss_cls: 0.7310  loss_bbox: 2.4524  loss_obj: 1.5788
03/19 18:36:38 - mmengine - INFO - Epoch(train)  [11][ 800/1000]  base_lr: 3.5347e-03 lr: 3.5347e-03  eta: 5:25:03  time: 0.2049  data_time: 0.0089  memory: 3937  loss: 4.8650  loss_cls: 0.7500  loss_bbox: 2.4947  loss_obj: 1.6203
03/19 18:36:49 - mmengine - INFO - Epoch(train)  [11][ 850/1000]  base_lr: 3.5126e-03 lr: 3.5126e-03  eta: 5:24:50  time: 0.2136  data_time: 0.0087  memory: 3656  loss: 4.9241  loss_cls: 0.7551  loss_bbox: 2.5235  loss_obj: 1.6455
03/19 18:36:59 - mmengine - INFO - Epoch(train)  [11][ 900/1000]  base_lr: 3.4904e-03 lr: 3.4904e-03  eta: 5:24:32  time: 0.2019  data_time: 0.0089  memory: 3357  loss: 4.8143  loss_cls: 0.7447  loss_bbox: 2.4764  loss_obj: 1.5932
03/19 18:37:10 - mmengine - INFO - Epoch(train)  [11][ 950/1000]  base_lr: 3.4681e-03 lr: 3.4681e-03  eta: 5:24:22  time: 0.2200  data_time: 0.0086  memory: 3639  loss: 4.7347  loss_cls: 0.7231  loss_bbox: 2.4459  loss_obj: 1.5657
03/19 18:37:21 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:37:21 - mmengine - INFO - Epoch(train)  [11][1000/1000]  base_lr: 3.4457e-03 lr: 3.4457e-03  eta: 5:24:14  time: 0.2263  data_time: 0.0086  memory: 3937  loss: 4.9124  loss_cls: 0.7503  loss_bbox: 2.4853  loss_obj: 1.6768
03/19 18:37:21 - mmengine - INFO - Saving checkpoint at 11 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:37:27 - mmengine - INFO - Epoch(val)  [11][ 50/250]    eta: 0:00:12  time: 0.0602  data_time: 0.0072  memory: 527  
03/19 18:37:30 - mmengine - INFO - Epoch(val)  [11][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 18:37:33 - mmengine - INFO - Epoch(val)  [11][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:37:35 - mmengine - INFO - Epoch(val)  [11][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0066  memory: 527  
03/19 18:37:38 - mmengine - INFO - Epoch(val)  [11][250/250]    eta: 0:00:00  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 18:37:40 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.32s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.83s).
Accumulating evaluation results...
DONE (t=2.93s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.170
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.421
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.110
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.103
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.217
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.317
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.286
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.286
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.381
03/19 18:37:54 - mmengine - INFO - bbox_mAP_copypaste: 0.170 0.421 0.110 0.103 0.217 0.317
03/19 18:37:54 - mmengine - INFO - Epoch(val) [11][250/250]    coco/bbox_mAP: 0.1700  coco/bbox_mAP_50: 0.4210  coco/bbox_mAP_75: 0.1100  coco/bbox_mAP_s: 0.1030  coco/bbox_mAP_m: 0.2170  coco/bbox_mAP_l: 0.3170  data_time: 0.0067  time: 0.0590
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:38:05 - mmengine - INFO - Epoch(train)  [12][  50/1000]  base_lr: 3.4233e-03 lr: 3.4233e-03  eta: 5:24:06  time: 0.2270  data_time: 0.0199  memory: 3937  loss: 4.8968  loss_cls: 0.7539  loss_bbox: 2.5197  loss_obj: 1.6232
03/19 18:38:16 - mmengine - INFO - Epoch(train)  [12][ 100/1000]  base_lr: 3.4008e-03 lr: 3.4008e-03  eta: 5:23:51  time: 0.2081  data_time: 0.0089  memory: 3937  loss: 4.8945  loss_cls: 0.7609  loss_bbox: 2.5005  loss_obj: 1.6331
03/19 18:38:28 - mmengine - INFO - Epoch(train)  [12][ 150/1000]  base_lr: 3.3782e-03 lr: 3.3782e-03  eta: 5:23:47  time: 0.2352  data_time: 0.0090  memory: 3937  loss: 4.7541  loss_cls: 0.7368  loss_bbox: 2.4407  loss_obj: 1.5766
03/19 18:38:38 - mmengine - INFO - Epoch(train)  [12][ 200/1000]  base_lr: 3.3555e-03 lr: 3.3555e-03  eta: 5:23:27  time: 0.1965  data_time: 0.0088  memory: 3071  loss: 4.7178  loss_cls: 0.7349  loss_bbox: 2.4567  loss_obj: 1.5262
03/19 18:38:49 - mmengine - INFO - Epoch(train)  [12][ 250/1000]  base_lr: 3.3328e-03 lr: 3.3328e-03  eta: 5:23:19  time: 0.2265  data_time: 0.0087  memory: 3357  loss: 4.7548  loss_cls: 0.7336  loss_bbox: 2.4450  loss_obj: 1.5763
03/19 18:39:00 - mmengine - INFO - Epoch(train)  [12][ 300/1000]  base_lr: 3.3100e-03 lr: 3.3100e-03  eta: 5:23:07  time: 0.2157  data_time: 0.0088  memory: 3639  loss: 4.8085  loss_cls: 0.7535  loss_bbox: 2.4746  loss_obj: 1.5803
03/19 18:39:11 - mmengine - INFO - Epoch(train)  [12][ 350/1000]  base_lr: 3.2872e-03 lr: 3.2872e-03  eta: 5:23:00  time: 0.2265  data_time: 0.0087  memory: 3937  loss: 4.8053  loss_cls: 0.7499  loss_bbox: 2.4449  loss_obj: 1.6105
03/19 18:39:22 - mmengine - INFO - Epoch(train)  [12][ 400/1000]  base_lr: 3.2642e-03 lr: 3.2642e-03  eta: 5:22:50  time: 0.2211  data_time: 0.0089  memory: 3937  loss: 4.8083  loss_cls: 0.7321  loss_bbox: 2.4775  loss_obj: 1.5987
03/19 18:39:33 - mmengine - INFO - Epoch(train)  [12][ 450/1000]  base_lr: 3.2413e-03 lr: 3.2413e-03  eta: 5:22:35  time: 0.2088  data_time: 0.0086  memory: 3071  loss: 4.8288  loss_cls: 0.7526  loss_bbox: 2.4795  loss_obj: 1.5966
03/19 18:39:42 - mmengine - INFO - Epoch(train)  [12][ 500/1000]  base_lr: 3.2183e-03 lr: 3.2183e-03  eta: 5:22:17  time: 0.1993  data_time: 0.0088  memory: 3071  loss: 4.7487  loss_cls: 0.7390  loss_bbox: 2.4474  loss_obj: 1.5623
03/19 18:39:53 - mmengine - INFO - Epoch(train)  [12][ 550/1000]  base_lr: 3.1952e-03 lr: 3.1952e-03  eta: 5:22:04  time: 0.2143  data_time: 0.0088  memory: 3639  loss: 4.7747  loss_cls: 0.7374  loss_bbox: 2.4643  loss_obj: 1.5729
03/19 18:40:03 - mmengine - INFO - Epoch(train)  [12][ 600/1000]  base_lr: 3.1721e-03 lr: 3.1721e-03  eta: 5:21:48  time: 0.2043  data_time: 0.0087  memory: 3639  loss: 4.8012  loss_cls: 0.7479  loss_bbox: 2.5003  loss_obj: 1.5531
03/19 18:40:15 - mmengine - INFO - Epoch(train)  [12][ 650/1000]  base_lr: 3.1489e-03 lr: 3.1489e-03  eta: 5:21:40  time: 0.2276  data_time: 0.0086  memory: 3639  loss: 4.8402  loss_cls: 0.7558  loss_bbox: 2.4751  loss_obj: 1.6093
03/19 18:40:24 - mmengine - INFO - Epoch(train)  [12][ 700/1000]  base_lr: 3.1257e-03 lr: 3.1257e-03  eta: 5:21:20  time: 0.1925  data_time: 0.0089  memory: 3357  loss: 4.8039  loss_cls: 0.7457  loss_bbox: 2.4862  loss_obj: 1.5720
03/19 18:40:35 - mmengine - INFO - Epoch(train)  [12][ 750/1000]  base_lr: 3.1024e-03 lr: 3.1024e-03  eta: 5:21:07  time: 0.2130  data_time: 0.0090  memory: 3639  loss: 4.8111  loss_cls: 0.7461  loss_bbox: 2.4717  loss_obj: 1.5932
03/19 18:40:46 - mmengine - INFO - Epoch(train)  [12][ 800/1000]  base_lr: 3.0792e-03 lr: 3.0792e-03  eta: 5:20:52  time: 0.2096  data_time: 0.0088  memory: 3357  loss: 4.7924  loss_cls: 0.7461  loss_bbox: 2.4747  loss_obj: 1.5716
03/19 18:40:56 - mmengine - INFO - Epoch(train)  [12][ 850/1000]  base_lr: 3.0558e-03 lr: 3.0558e-03  eta: 5:20:35  time: 0.2012  data_time: 0.0089  memory: 2817  loss: 4.7310  loss_cls: 0.7426  loss_bbox: 2.4616  loss_obj: 1.5268
03/19 18:41:07 - mmengine - INFO - Epoch(train)  [12][ 900/1000]  base_lr: 3.0325e-03 lr: 3.0325e-03  eta: 5:20:26  time: 0.2223  data_time: 0.0088  memory: 3937  loss: 4.7337  loss_cls: 0.7330  loss_bbox: 2.4533  loss_obj: 1.5474
03/19 18:41:17 - mmengine - INFO - Epoch(train)  [12][ 950/1000]  base_lr: 3.0091e-03 lr: 3.0091e-03  eta: 5:20:12  time: 0.2108  data_time: 0.0086  memory: 3357  loss: 4.7744  loss_cls: 0.7410  loss_bbox: 2.4712  loss_obj: 1.5622
03/19 18:41:28 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:41:28 - mmengine - INFO - Epoch(train)  [12][1000/1000]  base_lr: 2.9857e-03 lr: 2.9857e-03  eta: 5:19:59  time: 0.2128  data_time: 0.0086  memory: 3357  loss: 4.7442  loss_cls: 0.7320  loss_bbox: 2.4649  loss_obj: 1.5474
03/19 18:41:28 - mmengine - INFO - Saving checkpoint at 12 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:41:33 - mmengine - INFO - Epoch(val)  [12][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0071  memory: 527  
03/19 18:41:36 - mmengine - INFO - Epoch(val)  [12][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:41:39 - mmengine - INFO - Epoch(val)  [12][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:41:42 - mmengine - INFO - Epoch(val)  [12][200/250]    eta: 0:00:02  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 18:41:45 - mmengine - INFO - Epoch(val)  [12][250/250]    eta: 0:00:00  time: 0.0569  data_time: 0.0065  memory: 527  
03/19 18:41:47 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.32s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=10.09s).
Accumulating evaluation results...
DONE (t=3.05s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.176
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.428
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.114
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.102
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.245
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.325
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.295
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.295
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.295
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.228
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.442
03/19 18:42:01 - mmengine - INFO - bbox_mAP_copypaste: 0.176 0.428 0.114 0.102 0.245 0.325
03/19 18:42:01 - mmengine - INFO - Epoch(val) [12][250/250]    coco/bbox_mAP: 0.1760  coco/bbox_mAP_50: 0.4280  coco/bbox_mAP_75: 0.1140  coco/bbox_mAP_s: 0.1020  coco/bbox_mAP_m: 0.2450  coco/bbox_mAP_l: 0.3250  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:42:14 - mmengine - INFO - Epoch(train)  [13][  50/1000]  base_lr: 2.9622e-03 lr: 2.9622e-03  eta: 5:20:02  time: 0.2558  data_time: 0.0194  memory: 3937  loss: 4.8024  loss_cls: 0.7257  loss_bbox: 2.4428  loss_obj: 1.6339
03/19 18:42:25 - mmengine - INFO - Epoch(train)  [13][ 100/1000]  base_lr: 2.9387e-03 lr: 2.9387e-03  eta: 5:19:51  time: 0.2169  data_time: 0.0087  memory: 3357  loss: 4.7570  loss_cls: 0.7438  loss_bbox: 2.4685  loss_obj: 1.5447
03/19 18:42:37 - mmengine - INFO - Epoch(train)  [13][ 150/1000]  base_lr: 2.9153e-03 lr: 2.9153e-03  eta: 5:19:48  time: 0.2418  data_time: 0.0087  memory: 3937  loss: 4.7149  loss_cls: 0.7327  loss_bbox: 2.4209  loss_obj: 1.5612
03/19 18:42:46 - mmengine - INFO - Epoch(train)  [13][ 200/1000]  base_lr: 2.8917e-03 lr: 2.8917e-03  eta: 5:19:26  time: 0.1863  data_time: 0.0088  memory: 2587  loss: 4.7987  loss_cls: 0.7495  loss_bbox: 2.5011  loss_obj: 1.5482
03/19 18:42:56 - mmengine - INFO - Epoch(train)  [13][ 250/1000]  base_lr: 2.8682e-03 lr: 2.8682e-03  eta: 5:19:09  time: 0.2010  data_time: 0.0087  memory: 2817  loss: 4.7798  loss_cls: 0.7280  loss_bbox: 2.4761  loss_obj: 1.5757
03/19 18:43:07 - mmengine - INFO - Epoch(train)  [13][ 300/1000]  base_lr: 2.8447e-03 lr: 2.8447e-03  eta: 5:18:58  time: 0.2188  data_time: 0.0088  memory: 3357  loss: 4.7897  loss_cls: 0.7380  loss_bbox: 2.4524  loss_obj: 1.5992
03/19 18:43:18 - mmengine - INFO - Epoch(train)  [13][ 350/1000]  base_lr: 2.8211e-03 lr: 2.8211e-03  eta: 5:18:48  time: 0.2203  data_time: 0.0088  memory: 3937  loss: 4.8177  loss_cls: 0.7501  loss_bbox: 2.4695  loss_obj: 1.5981
03/19 18:43:29 - mmengine - INFO - Epoch(train)  [13][ 400/1000]  base_lr: 2.7976e-03 lr: 2.7976e-03  eta: 5:18:36  time: 0.2154  data_time: 0.0087  memory: 3357  loss: 4.6715  loss_cls: 0.7273  loss_bbox: 2.4107  loss_obj: 1.5335
03/19 18:43:41 - mmengine - INFO - Epoch(train)  [13][ 450/1000]  base_lr: 2.7740e-03 lr: 2.7740e-03  eta: 5:18:30  time: 0.2331  data_time: 0.0087  memory: 3937  loss: 4.7743  loss_cls: 0.7411  loss_bbox: 2.4492  loss_obj: 1.5840
03/19 18:43:50 - mmengine - INFO - Epoch(train)  [13][ 500/1000]  base_lr: 2.7505e-03 lr: 2.7505e-03  eta: 5:18:08  time: 0.1862  data_time: 0.0088  memory: 2587  loss: 4.7020  loss_cls: 0.7348  loss_bbox: 2.4511  loss_obj: 1.5162
03/19 18:44:01 - mmengine - INFO - Epoch(train)  [13][ 550/1000]  base_lr: 2.7269e-03 lr: 2.7269e-03  eta: 5:17:56  time: 0.2141  data_time: 0.0087  memory: 3357  loss: 4.7929  loss_cls: 0.7424  loss_bbox: 2.4664  loss_obj: 1.5840
03/19 18:44:11 - mmengine - INFO - Epoch(train)  [13][ 600/1000]  base_lr: 2.7034e-03 lr: 2.7034e-03  eta: 5:17:42  time: 0.2106  data_time: 0.0086  memory: 3071  loss: 4.7019  loss_cls: 0.7334  loss_bbox: 2.4464  loss_obj: 1.5222
03/19 18:44:21 - mmengine - INFO - Epoch(train)  [13][ 650/1000]  base_lr: 2.6798e-03 lr: 2.6798e-03  eta: 5:17:25  time: 0.2006  data_time: 0.0088  memory: 3357  loss: 4.7567  loss_cls: 0.7289  loss_bbox: 2.4698  loss_obj: 1.5580
03/19 18:44:33 - mmengine - INFO - Epoch(train)  [13][ 700/1000]  base_lr: 2.6563e-03 lr: 2.6563e-03  eta: 5:17:18  time: 0.2286  data_time: 0.0087  memory: 3639  loss: 4.7272  loss_cls: 0.7264  loss_bbox: 2.4385  loss_obj: 1.5624
03/19 18:44:43 - mmengine - INFO - Epoch(train)  [13][ 750/1000]  base_lr: 2.6327e-03 lr: 2.6327e-03  eta: 5:17:02  time: 0.2043  data_time: 0.0089  memory: 3937  loss: 4.7491  loss_cls: 0.7328  loss_bbox: 2.4753  loss_obj: 1.5409
03/19 18:44:54 - mmengine - INFO - Epoch(train)  [13][ 800/1000]  base_lr: 2.6092e-03 lr: 2.6092e-03  eta: 5:16:55  time: 0.2276  data_time: 0.0087  memory: 3937  loss: 4.7170  loss_cls: 0.7231  loss_bbox: 2.4504  loss_obj: 1.5435
03/19 18:45:05 - mmengine - INFO - Epoch(train)  [13][ 850/1000]  base_lr: 2.5857e-03 lr: 2.5857e-03  eta: 5:16:45  time: 0.2219  data_time: 0.0087  memory: 3639  loss: 4.7419  loss_cls: 0.7311  loss_bbox: 2.4552  loss_obj: 1.5555
03/19 18:45:17 - mmengine - INFO - Epoch(train)  [13][ 900/1000]  base_lr: 2.5622e-03 lr: 2.5622e-03  eta: 5:16:39  time: 0.2316  data_time: 0.0086  memory: 3937  loss: 4.7076  loss_cls: 0.7277  loss_bbox: 2.4341  loss_obj: 1.5458
03/19 18:45:29 - mmengine - INFO - Epoch(train)  [13][ 950/1000]  base_lr: 2.5387e-03 lr: 2.5387e-03  eta: 5:16:31  time: 0.2291  data_time: 0.0087  memory: 3937  loss: 4.6417  loss_cls: 0.7236  loss_bbox: 2.4092  loss_obj: 1.5089
03/19 18:45:39 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:45:39 - mmengine - INFO - Epoch(train)  [13][1000/1000]  base_lr: 2.5153e-03 lr: 2.5153e-03  eta: 5:16:19  time: 0.2128  data_time: 0.0087  memory: 3937  loss: 4.6625  loss_cls: 0.7319  loss_bbox: 2.3983  loss_obj: 1.5323
03/19 18:45:39 - mmengine - INFO - Saving checkpoint at 13 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:45:45 - mmengine - INFO - Epoch(val)  [13][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0071  memory: 527  
03/19 18:45:47 - mmengine - INFO - Epoch(val)  [13][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 18:45:50 - mmengine - INFO - Epoch(val)  [13][150/250]    eta: 0:00:05  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 18:45:53 - mmengine - INFO - Epoch(val)  [13][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 18:45:56 - mmengine - INFO - Epoch(val)  [13][250/250]    eta: 0:00:00  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 18:45:58 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.32s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.65s).
Accumulating evaluation results...
DONE (t=2.83s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.181
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.445
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.116
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.109
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.246
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.299
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.299
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.299
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.359
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.422
03/19 18:46:12 - mmengine - INFO - bbox_mAP_copypaste: 0.181 0.445 0.116 0.109 0.246 0.322
03/19 18:46:12 - mmengine - INFO - Epoch(val) [13][250/250]    coco/bbox_mAP: 0.1810  coco/bbox_mAP_50: 0.4450  coco/bbox_mAP_75: 0.1160  coco/bbox_mAP_s: 0.1090  coco/bbox_mAP_m: 0.2460  coco/bbox_mAP_l: 0.3220  data_time: 0.0066  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:46:22 - mmengine - INFO - Epoch(train)  [14][  50/1000]  base_lr: 2.4919e-03 lr: 2.4919e-03  eta: 5:16:04  time: 0.2071  data_time: 0.0200  memory: 2817  loss: 4.6309  loss_cls: 0.7227  loss_bbox: 2.4273  loss_obj: 1.4810
03/19 18:46:32 - mmengine - INFO - Epoch(train)  [14][ 100/1000]  base_lr: 2.4685e-03 lr: 2.4685e-03  eta: 5:15:46  time: 0.1959  data_time: 0.0087  memory: 2587  loss: 4.7415  loss_cls: 0.7358  loss_bbox: 2.4526  loss_obj: 1.5530
03/19 18:46:43 - mmengine - INFO - Epoch(train)  [14][ 150/1000]  base_lr: 2.4451e-03 lr: 2.4451e-03  eta: 5:15:38  time: 0.2275  data_time: 0.0087  memory: 3639  loss: 4.6281  loss_cls: 0.7229  loss_bbox: 2.3977  loss_obj: 1.5074
03/19 18:46:54 - mmengine - INFO - Epoch(train)  [14][ 200/1000]  base_lr: 2.4218e-03 lr: 2.4218e-03  eta: 5:15:26  time: 0.2133  data_time: 0.0087  memory: 3639  loss: 4.6852  loss_cls: 0.7293  loss_bbox: 2.4369  loss_obj: 1.5190
03/19 18:47:05 - mmengine - INFO - Epoch(train)  [14][ 250/1000]  base_lr: 2.3985e-03 lr: 2.3985e-03  eta: 5:15:16  time: 0.2230  data_time: 0.0087  memory: 3639  loss: 4.6660  loss_cls: 0.7280  loss_bbox: 2.4085  loss_obj: 1.5295
03/19 18:47:15 - mmengine - INFO - Epoch(train)  [14][ 300/1000]  base_lr: 2.3752e-03 lr: 2.3752e-03  eta: 5:15:02  time: 0.2065  data_time: 0.0089  memory: 3639  loss: 4.6924  loss_cls: 0.7223  loss_bbox: 2.4434  loss_obj: 1.5267
03/19 18:47:26 - mmengine - INFO - Epoch(train)  [14][ 350/1000]  base_lr: 2.3520e-03 lr: 2.3520e-03  eta: 5:14:47  time: 0.2065  data_time: 0.0087  memory: 2817  loss: 4.6095  loss_cls: 0.7097  loss_bbox: 2.4079  loss_obj: 1.4919
03/19 18:47:37 - mmengine - INFO - Epoch(train)  [14][ 400/1000]  base_lr: 2.3289e-03 lr: 2.3289e-03  eta: 5:14:39  time: 0.2265  data_time: 0.0088  memory: 3937  loss: 4.7447  loss_cls: 0.7255  loss_bbox: 2.4421  loss_obj: 1.5770
03/19 18:47:48 - mmengine - INFO - Epoch(train)  [14][ 450/1000]  base_lr: 2.3057e-03 lr: 2.3057e-03  eta: 5:14:31  time: 0.2287  data_time: 0.0086  memory: 3937  loss: 4.6148  loss_cls: 0.7106  loss_bbox: 2.4024  loss_obj: 1.5018
03/19 18:47:59 - mmengine - INFO - Epoch(train)  [14][ 500/1000]  base_lr: 2.2827e-03 lr: 2.2827e-03  eta: 5:14:17  time: 0.2083  data_time: 0.0087  memory: 2817  loss: 4.6812  loss_cls: 0.7336  loss_bbox: 2.4303  loss_obj: 1.5172
03/19 18:48:09 - mmengine - INFO - Epoch(train)  [14][ 550/1000]  base_lr: 2.2596e-03 lr: 2.2596e-03  eta: 5:14:02  time: 0.2023  data_time: 0.0087  memory: 3639  loss: 4.6821  loss_cls: 0.7302  loss_bbox: 2.4404  loss_obj: 1.5116
03/19 18:48:19 - mmengine - INFO - Epoch(train)  [14][ 600/1000]  base_lr: 2.2367e-03 lr: 2.2367e-03  eta: 5:13:45  time: 0.2006  data_time: 0.0088  memory: 3071  loss: 4.7210  loss_cls: 0.7378  loss_bbox: 2.4595  loss_obj: 1.5237
03/19 18:48:29 - mmengine - INFO - Epoch(train)  [14][ 650/1000]  base_lr: 2.2138e-03 lr: 2.2138e-03  eta: 5:13:26  time: 0.1907  data_time: 0.0086  memory: 2337  loss: 4.6737  loss_cls: 0.7389  loss_bbox: 2.4226  loss_obj: 1.5122
03/19 18:48:40 - mmengine - INFO - Epoch(train)  [14][ 700/1000]  base_lr: 2.1909e-03 lr: 2.1909e-03  eta: 5:13:15  time: 0.2191  data_time: 0.0087  memory: 3639  loss: 4.6879  loss_cls: 0.7307  loss_bbox: 2.4080  loss_obj: 1.5492
03/19 18:48:51 - mmengine - INFO - Epoch(train)  [14][ 750/1000]  base_lr: 2.1681e-03 lr: 2.1681e-03  eta: 5:13:08  time: 0.2305  data_time: 0.0086  memory: 3639  loss: 4.6632  loss_cls: 0.7200  loss_bbox: 2.4134  loss_obj: 1.5299
03/19 18:49:00 - mmengine - INFO - Epoch(train)  [14][ 800/1000]  base_lr: 2.1454e-03 lr: 2.1454e-03  eta: 5:12:46  time: 0.1828  data_time: 0.0089  memory: 2337  loss: 4.7111  loss_cls: 0.7367  loss_bbox: 2.4448  loss_obj: 1.5297
03/19 18:49:11 - mmengine - INFO - Epoch(train)  [14][ 850/1000]  base_lr: 2.1227e-03 lr: 2.1227e-03  eta: 5:12:33  time: 0.2104  data_time: 0.0089  memory: 3071  loss: 4.6815  loss_cls: 0.7311  loss_bbox: 2.4186  loss_obj: 1.5317
03/19 18:49:22 - mmengine - INFO - Epoch(train)  [14][ 900/1000]  base_lr: 2.1001e-03 lr: 2.1001e-03  eta: 5:12:26  time: 0.2294  data_time: 0.0087  memory: 3639  loss: 4.6255  loss_cls: 0.7197  loss_bbox: 2.3901  loss_obj: 1.5157
03/19 18:49:33 - mmengine - INFO - Epoch(train)  [14][ 950/1000]  base_lr: 2.0776e-03 lr: 2.0776e-03  eta: 5:12:15  time: 0.2159  data_time: 0.0087  memory: 3071  loss: 4.6727  loss_cls: 0.7230  loss_bbox: 2.4227  loss_obj: 1.5270
03/19 18:49:44 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:49:44 - mmengine - INFO - Epoch(train)  [14][1000/1000]  base_lr: 2.0552e-03 lr: 2.0552e-03  eta: 5:12:02  time: 0.2122  data_time: 0.0085  memory: 3639  loss: 4.6730  loss_cls: 0.7275  loss_bbox: 2.4246  loss_obj: 1.5209
03/19 18:49:44 - mmengine - INFO - Saving checkpoint at 14 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:49:49 - mmengine - INFO - Epoch(val)  [14][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0070  memory: 527  
03/19 18:49:52 - mmengine - INFO - Epoch(val)  [14][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:49:55 - mmengine - INFO - Epoch(val)  [14][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:49:58 - mmengine - INFO - Epoch(val)  [14][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 18:50:01 - mmengine - INFO - Epoch(val)  [14][250/250]    eta: 0:00:00  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 18:50:03 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.31s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.77s).
Accumulating evaluation results...
DONE (t=2.90s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.185
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.451
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.115
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.110
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.245
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.331
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.303
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.303
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.303
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.236
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.423
03/19 18:50:16 - mmengine - INFO - bbox_mAP_copypaste: 0.185 0.451 0.115 0.110 0.245 0.331
03/19 18:50:16 - mmengine - INFO - Epoch(val) [14][250/250]    coco/bbox_mAP: 0.1850  coco/bbox_mAP_50: 0.4510  coco/bbox_mAP_75: 0.1150  coco/bbox_mAP_s: 0.1100  coco/bbox_mAP_m: 0.2450  coco/bbox_mAP_l: 0.3310  data_time: 0.0066  time: 0.0588
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:50:28 - mmengine - INFO - Epoch(train)  [15][  50/1000]  base_lr: 2.0328e-03 lr: 2.0328e-03  eta: 5:11:54  time: 0.2287  data_time: 0.0191  memory: 3357  loss: 4.7036  loss_cls: 0.7251  loss_bbox: 2.4188  loss_obj: 1.5598
03/19 18:50:39 - mmengine - INFO - Epoch(train)  [15][ 100/1000]  base_lr: 2.0105e-03 lr: 2.0105e-03  eta: 5:11:46  time: 0.2258  data_time: 0.0087  memory: 3937  loss: 4.6285  loss_cls: 0.7185  loss_bbox: 2.3951  loss_obj: 1.5150
03/19 18:50:50 - mmengine - INFO - Epoch(train)  [15][ 150/1000]  base_lr: 1.9883e-03 lr: 1.9883e-03  eta: 5:11:36  time: 0.2196  data_time: 0.0087  memory: 3937  loss: 4.6907  loss_cls: 0.7328  loss_bbox: 2.4322  loss_obj: 1.5258
03/19 18:51:01 - mmengine - INFO - Epoch(train)  [15][ 200/1000]  base_lr: 1.9662e-03 lr: 1.9662e-03  eta: 5:11:23  time: 0.2117  data_time: 0.0087  memory: 3357  loss: 4.5282  loss_cls: 0.7118  loss_bbox: 2.3987  loss_obj: 1.4177
03/19 18:51:11 - mmengine - INFO - Epoch(train)  [15][ 250/1000]  base_lr: 1.9441e-03 lr: 1.9441e-03  eta: 5:11:10  time: 0.2094  data_time: 0.0088  memory: 3937  loss: 4.6012  loss_cls: 0.7236  loss_bbox: 2.4139  loss_obj: 1.4637
03/19 18:51:22 - mmengine - INFO - Epoch(train)  [15][ 300/1000]  base_lr: 1.9222e-03 lr: 1.9222e-03  eta: 5:10:56  time: 0.2090  data_time: 0.0087  memory: 3357  loss: 4.6476  loss_cls: 0.7287  loss_bbox: 2.4174  loss_obj: 1.5015
03/19 18:51:33 - mmengine - INFO - Epoch(train)  [15][ 350/1000]  base_lr: 1.9003e-03 lr: 1.9003e-03  eta: 5:10:51  time: 0.2368  data_time: 0.0088  memory: 3937  loss: 4.6801  loss_cls: 0.7218  loss_bbox: 2.4114  loss_obj: 1.5469
03/19 18:51:43 - mmengine - INFO - Epoch(train)  [15][ 400/1000]  base_lr: 1.8785e-03 lr: 1.8785e-03  eta: 5:10:34  time: 0.1981  data_time: 0.0087  memory: 3071  loss: 4.6735  loss_cls: 0.7336  loss_bbox: 2.4485  loss_obj: 1.4915
03/19 18:51:55 - mmengine - INFO - Epoch(train)  [15][ 450/1000]  base_lr: 1.8568e-03 lr: 1.8568e-03  eta: 5:10:25  time: 0.2250  data_time: 0.0086  memory: 3639  loss: 4.6331  loss_cls: 0.7185  loss_bbox: 2.3903  loss_obj: 1.5243
03/19 18:52:07 - mmengine - INFO - Epoch(train)  [15][ 500/1000]  base_lr: 1.8353e-03 lr: 1.8353e-03  eta: 5:10:26  time: 0.2561  data_time: 0.0086  memory: 3937  loss: 4.6493  loss_cls: 0.7200  loss_bbox: 2.3992  loss_obj: 1.5301
03/19 18:52:18 - mmengine - INFO - Epoch(train)  [15][ 550/1000]  base_lr: 1.8138e-03 lr: 1.8138e-03  eta: 5:10:16  time: 0.2204  data_time: 0.0086  memory: 3937  loss: 4.6792  loss_cls: 0.7333  loss_bbox: 2.4248  loss_obj: 1.5212
03/19 18:52:29 - mmengine - INFO - Epoch(train)  [15][ 600/1000]  base_lr: 1.7924e-03 lr: 1.7924e-03  eta: 5:10:04  time: 0.2150  data_time: 0.0087  memory: 3937  loss: 4.5031  loss_cls: 0.6828  loss_bbox: 2.3774  loss_obj: 1.4429
03/19 18:52:40 - mmengine - INFO - Epoch(train)  [15][ 650/1000]  base_lr: 1.7712e-03 lr: 1.7712e-03  eta: 5:09:54  time: 0.2217  data_time: 0.0087  memory: 3937  loss: 4.5722  loss_cls: 0.7115  loss_bbox: 2.3843  loss_obj: 1.4764
03/19 18:52:52 - mmengine - INFO - Epoch(train)  [15][ 700/1000]  base_lr: 1.7500e-03 lr: 1.7500e-03  eta: 5:09:50  time: 0.2419  data_time: 0.0088  memory: 3937  loss: 4.6077  loss_cls: 0.7125  loss_bbox: 2.3916  loss_obj: 1.5035
03/19 18:53:04 - mmengine - INFO - Epoch(train)  [15][ 750/1000]  base_lr: 1.7289e-03 lr: 1.7289e-03  eta: 5:09:40  time: 0.2214  data_time: 0.0089  memory: 3937  loss: 4.6440  loss_cls: 0.7198  loss_bbox: 2.4212  loss_obj: 1.5030
03/19 18:53:14 - mmengine - INFO - Epoch(train)  [15][ 800/1000]  base_lr: 1.7080e-03 lr: 1.7080e-03  eta: 5:09:28  time: 0.2120  data_time: 0.0087  memory: 3357  loss: 4.6426  loss_cls: 0.7252  loss_bbox: 2.4183  loss_obj: 1.4991
03/19 18:53:26 - mmengine - INFO - Epoch(train)  [15][ 850/1000]  base_lr: 1.6872e-03 lr: 1.6872e-03  eta: 5:09:22  time: 0.2361  data_time: 0.0087  memory: 3937  loss: 4.5745  loss_cls: 0.7053  loss_bbox: 2.3703  loss_obj: 1.4989
03/19 18:53:36 - mmengine - INFO - Epoch(train)  [15][ 900/1000]  base_lr: 1.6665e-03 lr: 1.6665e-03  eta: 5:09:08  time: 0.2054  data_time: 0.0085  memory: 3071  loss: 4.6085  loss_cls: 0.7197  loss_bbox: 2.4140  loss_obj: 1.4748
03/19 18:53:48 - mmengine - INFO - Epoch(train)  [15][ 950/1000]  base_lr: 1.6459e-03 lr: 1.6459e-03  eta: 5:09:03  time: 0.2417  data_time: 0.0088  memory: 3937  loss: 4.6628  loss_cls: 0.7310  loss_bbox: 2.4301  loss_obj: 1.5016
03/19 18:53:59 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:53:59 - mmengine - INFO - Epoch(train)  [15][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:08:50  time: 0.2101  data_time: 0.0087  memory: 3357  loss: 4.5805  loss_cls: 0.7091  loss_bbox: 2.4064  loss_obj: 1.4651
03/19 18:53:59 - mmengine - INFO - Saving checkpoint at 15 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:54:04 - mmengine - INFO - Epoch(val)  [15][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0071  memory: 527  
03/19 18:54:07 - mmengine - INFO - Epoch(val)  [15][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 18:54:10 - mmengine - INFO - Epoch(val)  [15][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 18:54:13 - mmengine - INFO - Epoch(val)  [15][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0064  memory: 527  
03/19 18:54:16 - mmengine - INFO - Epoch(val)  [15][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0065  memory: 527  
03/19 18:54:18 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.31s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.57s).
Accumulating evaluation results...
DONE (t=2.83s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.187
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.455
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.121
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.111
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.251
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.343
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.302
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.302
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.302
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.237
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.362
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.431
03/19 18:54:31 - mmengine - INFO - bbox_mAP_copypaste: 0.187 0.455 0.121 0.111 0.251 0.343
03/19 18:54:31 - mmengine - INFO - Epoch(val) [15][250/250]    coco/bbox_mAP: 0.1870  coco/bbox_mAP_50: 0.4550  coco/bbox_mAP_75: 0.1210  coco/bbox_mAP_s: 0.1110  coco/bbox_mAP_m: 0.2510  coco/bbox_mAP_l: 0.3430  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:54:44 - mmengine - INFO - Epoch(train)  [16][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:08:52  time: 0.2641  data_time: 0.0185  memory: 3937  loss: 4.6516  loss_cls: 0.7112  loss_bbox: 2.4013  loss_obj: 1.5391
03/19 18:54:56 - mmengine - INFO - Epoch(train)  [16][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:08:45  time: 0.2300  data_time: 0.0079  memory: 3639  loss: 4.5264  loss_cls: 0.7076  loss_bbox: 2.3778  loss_obj: 1.4410
03/19 18:55:06 - mmengine - INFO - Epoch(train)  [16][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:08:31  time: 0.2075  data_time: 0.0081  memory: 3357  loss: 4.5912  loss_cls: 0.7144  loss_bbox: 2.4116  loss_obj: 1.4652
03/19 18:55:16 - mmengine - INFO - Epoch(train)  [16][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:08:16  time: 0.2020  data_time: 0.0082  memory: 3639  loss: 4.6130  loss_cls: 0.7180  loss_bbox: 2.4294  loss_obj: 1.4657
03/19 18:55:26 - mmengine - INFO - Epoch(train)  [16][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:07:56  time: 0.1880  data_time: 0.0081  memory: 2817  loss: 4.5504  loss_cls: 0.7246  loss_bbox: 2.4130  loss_obj: 1.4128
03/19 18:55:36 - mmengine - INFO - Epoch(train)  [16][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:07:44  time: 0.2121  data_time: 0.0079  memory: 3357  loss: 4.5454  loss_cls: 0.7100  loss_bbox: 2.3804  loss_obj: 1.4551
03/19 18:55:46 - mmengine - INFO - Epoch(train)  [16][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:07:27  time: 0.1961  data_time: 0.0081  memory: 2817  loss: 4.6166  loss_cls: 0.7177  loss_bbox: 2.4285  loss_obj: 1.4704
03/19 18:55:56 - mmengine - INFO - Epoch(train)  [16][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:07:09  time: 0.1922  data_time: 0.0081  memory: 3071  loss: 4.6060  loss_cls: 0.7338  loss_bbox: 2.4185  loss_obj: 1.4536
03/19 18:56:07 - mmengine - INFO - Epoch(train)  [16][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:07:00  time: 0.2237  data_time: 0.0080  memory: 3639  loss: 4.6181  loss_cls: 0.7106  loss_bbox: 2.4250  loss_obj: 1.4825
03/19 18:56:18 - mmengine - INFO - Epoch(train)  [16][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:06:52  time: 0.2298  data_time: 0.0080  memory: 3937  loss: 4.6509  loss_cls: 0.7266  loss_bbox: 2.3990  loss_obj: 1.5252
03/19 18:56:29 - mmengine - INFO - Epoch(train)  [16][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:06:41  time: 0.2164  data_time: 0.0080  memory: 3937  loss: 4.6206  loss_cls: 0.7173  loss_bbox: 2.4230  loss_obj: 1.4803
03/19 18:56:39 - mmengine - INFO - Epoch(train)  [16][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:06:23  time: 0.1934  data_time: 0.0082  memory: 3071  loss: 4.5552  loss_cls: 0.7165  loss_bbox: 2.4085  loss_obj: 1.4302
03/19 18:56:50 - mmengine - INFO - Epoch(train)  [16][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:06:13  time: 0.2206  data_time: 0.0079  memory: 3357  loss: 4.6374  loss_cls: 0.7170  loss_bbox: 2.4190  loss_obj: 1.5013
03/19 18:57:02 - mmengine - INFO - Epoch(train)  [16][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:06:08  time: 0.2418  data_time: 0.0080  memory: 3937  loss: 4.4989  loss_cls: 0.6980  loss_bbox: 2.3558  loss_obj: 1.4450
03/19 18:57:13 - mmengine - INFO - Epoch(train)  [16][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:05:55  time: 0.2091  data_time: 0.0081  memory: 3071  loss: 4.6132  loss_cls: 0.7210  loss_bbox: 2.3925  loss_obj: 1.4997
03/19 18:57:23 - mmengine - INFO - Epoch(train)  [16][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:05:44  time: 0.2175  data_time: 0.0081  memory: 3937  loss: 4.6057  loss_cls: 0.7161  loss_bbox: 2.4044  loss_obj: 1.4853
03/19 18:57:34 - mmengine - INFO - Epoch(train)  [16][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:05:29  time: 0.2005  data_time: 0.0080  memory: 3639  loss: 4.6322  loss_cls: 0.7209  loss_bbox: 2.4256  loss_obj: 1.4857
03/19 18:57:46 - mmengine - INFO - Epoch(train)  [16][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:05:24  time: 0.2428  data_time: 0.0080  memory: 3937  loss: 4.5781  loss_cls: 0.7084  loss_bbox: 2.3667  loss_obj: 1.5030
03/19 18:57:57 - mmengine - INFO - Epoch(train)  [16][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:05:13  time: 0.2166  data_time: 0.0081  memory: 3937  loss: 4.6240  loss_cls: 0.7136  loss_bbox: 2.4144  loss_obj: 1.4961
03/19 18:58:06 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 18:58:06 - mmengine - INFO - Epoch(train)  [16][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:57  time: 0.1974  data_time: 0.0079  memory: 2817  loss: 4.4998  loss_cls: 0.7030  loss_bbox: 2.3812  loss_obj: 1.4155
03/19 18:58:06 - mmengine - INFO - Saving checkpoint at 16 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:58:12 - mmengine - INFO - Epoch(val)  [16][ 50/250]    eta: 0:00:11  time: 0.0587  data_time: 0.0070  memory: 527  
03/19 18:58:15 - mmengine - INFO - Epoch(val)  [16][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 18:58:18 - mmengine - INFO - Epoch(val)  [16][150/250]    eta: 0:00:05  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 18:58:21 - mmengine - INFO - Epoch(val)  [16][200/250]    eta: 0:00:02  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 18:58:23 - mmengine - INFO - Epoch(val)  [16][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0065  memory: 527  
03/19 18:58:25 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.32s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.61s).
Accumulating evaluation results...
DONE (t=2.83s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.189
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.458
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.123
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.112
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.252
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.354
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.300
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.300
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.300
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.234
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.361
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.432
03/19 18:58:39 - mmengine - INFO - bbox_mAP_copypaste: 0.189 0.458 0.123 0.112 0.252 0.354
03/19 18:58:39 - mmengine - INFO - Epoch(val) [16][250/250]    coco/bbox_mAP: 0.1890  coco/bbox_mAP_50: 0.4580  coco/bbox_mAP_75: 0.1230  coco/bbox_mAP_s: 0.1120  coco/bbox_mAP_m: 0.2520  coco/bbox_mAP_l: 0.3540  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 18:58:50 - mmengine - INFO - Epoch(train)  [17][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:49  time: 0.2279  data_time: 0.0185  memory: 3639  loss: 4.6362  loss_cls: 0.7136  loss_bbox: 2.4192  loss_obj: 1.5033
03/19 18:59:01 - mmengine - INFO - Epoch(train)  [17][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:38  time: 0.2176  data_time: 0.0081  memory: 3937  loss: 4.5477  loss_cls: 0.7108  loss_bbox: 2.3834  loss_obj: 1.4535
03/19 18:59:13 - mmengine - INFO - Epoch(train)  [17][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:32  time: 0.2379  data_time: 0.0079  memory: 3937  loss: 4.6367  loss_cls: 0.7214  loss_bbox: 2.4067  loss_obj: 1.5086
03/19 18:59:24 - mmengine - INFO - Epoch(train)  [17][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:23  time: 0.2235  data_time: 0.0082  memory: 3937  loss: 4.6002  loss_cls: 0.7155  loss_bbox: 2.4034  loss_obj: 1.4812
03/19 18:59:34 - mmengine - INFO - Epoch(train)  [17][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:04:06  time: 0.1978  data_time: 0.0080  memory: 3357  loss: 4.5808  loss_cls: 0.7256  loss_bbox: 2.4053  loss_obj: 1.4499
03/19 18:59:45 - mmengine - INFO - Epoch(train)  [17][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:55  time: 0.2159  data_time: 0.0082  memory: 3639  loss: 4.5985  loss_cls: 0.7284  loss_bbox: 2.4037  loss_obj: 1.4664
03/19 18:59:56 - mmengine - INFO - Epoch(train)  [17][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:47  time: 0.2292  data_time: 0.0079  memory: 3937  loss: 4.5265  loss_cls: 0.7072  loss_bbox: 2.3692  loss_obj: 1.4501
03/19 19:00:09 - mmengine - INFO - Epoch(train)  [17][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:43  time: 0.2444  data_time: 0.0079  memory: 3937  loss: 4.5848  loss_cls: 0.7082  loss_bbox: 2.3970  loss_obj: 1.4796
03/19 19:00:20 - mmengine - INFO - Epoch(train)  [17][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:35  time: 0.2295  data_time: 0.0081  memory: 3937  loss: 4.6070  loss_cls: 0.7192  loss_bbox: 2.4041  loss_obj: 1.4838
03/19 19:00:31 - mmengine - INFO - Epoch(train)  [17][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:24  time: 0.2165  data_time: 0.0081  memory: 3937  loss: 4.6377  loss_cls: 0.7240  loss_bbox: 2.4105  loss_obj: 1.5032
03/19 19:00:41 - mmengine - INFO - Epoch(train)  [17][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:03:08  time: 0.1972  data_time: 0.0080  memory: 2817  loss: 4.5686  loss_cls: 0.7197  loss_bbox: 2.3853  loss_obj: 1.4636
03/19 19:00:51 - mmengine - INFO - Epoch(train)  [17][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:53  time: 0.2050  data_time: 0.0081  memory: 3639  loss: 4.5313  loss_cls: 0.7099  loss_bbox: 2.3772  loss_obj: 1.4443
03/19 19:01:03 - mmengine - INFO - Epoch(train)  [17][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:48  time: 0.2397  data_time: 0.0080  memory: 3937  loss: 4.5315  loss_cls: 0.7019  loss_bbox: 2.3600  loss_obj: 1.4696
03/19 19:01:14 - mmengine - INFO - Epoch(train)  [17][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:38  time: 0.2208  data_time: 0.0079  memory: 3357  loss: 4.5547  loss_cls: 0.7012  loss_bbox: 2.3950  loss_obj: 1.4585
03/19 19:01:25 - mmengine - INFO - Epoch(train)  [17][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:29  time: 0.2246  data_time: 0.0080  memory: 3937  loss: 4.5434  loss_cls: 0.7074  loss_bbox: 2.3842  loss_obj: 1.4517
03/19 19:01:35 - mmengine - INFO - Epoch(train)  [17][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:12  time: 0.1939  data_time: 0.0082  memory: 3639  loss: 4.5104  loss_cls: 0.7070  loss_bbox: 2.3782  loss_obj: 1.4252
03/19 19:01:46 - mmengine - INFO - Epoch(train)  [17][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:02:02  time: 0.2233  data_time: 0.0080  memory: 3639  loss: 4.5873  loss_cls: 0.7191  loss_bbox: 2.3857  loss_obj: 1.4825
03/19 19:01:57 - mmengine - INFO - Epoch(train)  [17][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:01:53  time: 0.2255  data_time: 0.0080  memory: 3639  loss: 4.5453  loss_cls: 0.6907  loss_bbox: 2.3703  loss_obj: 1.4842
03/19 19:02:06 - mmengine - INFO - Epoch(train)  [17][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:01:33  time: 0.1798  data_time: 0.0082  memory: 2337  loss: 4.6372  loss_cls: 0.7293  loss_bbox: 2.4531  loss_obj: 1.4547
03/19 19:02:17 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:02:17 - mmengine - INFO - Epoch(train)  [17][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:01:18  time: 0.2009  data_time: 0.0080  memory: 3639  loss: 4.5700  loss_cls: 0.7024  loss_bbox: 2.4164  loss_obj: 1.4512
03/19 19:02:17 - mmengine - INFO - Saving checkpoint at 17 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:02:22 - mmengine - INFO - Epoch(val)  [17][ 50/250]    eta: 0:00:11  time: 0.0599  data_time: 0.0072  memory: 527  
03/19 19:02:25 - mmengine - INFO - Epoch(val)  [17][100/250]    eta: 0:00:08  time: 0.0593  data_time: 0.0065  memory: 527  
03/19 19:02:28 - mmengine - INFO - Epoch(val)  [17][150/250]    eta: 0:00:05  time: 0.0589  data_time: 0.0066  memory: 527  
03/19 19:02:31 - mmengine - INFO - Epoch(val)  [17][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 19:02:34 - mmengine - INFO - Epoch(val)  [17][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 19:02:35 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.31s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.53s).
Accumulating evaluation results...
DONE (t=2.78s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.192
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.459
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.127
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.114
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.256
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.301
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.301
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.301
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.235
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.364
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.424
03/19 19:02:49 - mmengine - INFO - bbox_mAP_copypaste: 0.192 0.459 0.127 0.114 0.256 0.349
03/19 19:02:49 - mmengine - INFO - Epoch(val) [17][250/250]    coco/bbox_mAP: 0.1920  coco/bbox_mAP_50: 0.4590  coco/bbox_mAP_75: 0.1270  coco/bbox_mAP_s: 0.1140  coco/bbox_mAP_m: 0.2560  coco/bbox_mAP_l: 0.3490  data_time: 0.0067  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:03:00 - mmengine - INFO - Epoch(train)  [18][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:01:09  time: 0.2273  data_time: 0.0178  memory: 3937  loss: 4.5822  loss_cls: 0.7192  loss_bbox: 2.4120  loss_obj: 1.4509
03/19 19:03:11 - mmengine - INFO - Epoch(train)  [18][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:01:00  time: 0.2263  data_time: 0.0080  memory: 3639  loss: 4.5063  loss_cls: 0.7071  loss_bbox: 2.3701  loss_obj: 1.4291
03/19 19:03:22 - mmengine - INFO - Epoch(train)  [18][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:00:49  time: 0.2147  data_time: 0.0081  memory: 3937  loss: 4.5656  loss_cls: 0.7110  loss_bbox: 2.3844  loss_obj: 1.4702
03/19 19:03:33 - mmengine - INFO - Epoch(train)  [18][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:00:40  time: 0.2249  data_time: 0.0080  memory: 3937  loss: 4.5863  loss_cls: 0.7171  loss_bbox: 2.4036  loss_obj: 1.4656
03/19 19:03:44 - mmengine - INFO - Epoch(train)  [18][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:00:28  time: 0.2135  data_time: 0.0080  memory: 3937  loss: 4.5706  loss_cls: 0.7092  loss_bbox: 2.3878  loss_obj: 1.4735
03/19 19:03:55 - mmengine - INFO - Epoch(train)  [18][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:00:15  time: 0.2121  data_time: 0.0080  memory: 3639  loss: 4.5026  loss_cls: 0.7096  loss_bbox: 2.3578  loss_obj: 1.4352
03/19 19:04:06 - mmengine - INFO - Epoch(train)  [18][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 5:00:04  time: 0.2168  data_time: 0.0080  memory: 3639  loss: 4.5290  loss_cls: 0.7106  loss_bbox: 2.3790  loss_obj: 1.4394
03/19 19:04:16 - mmengine - INFO - Epoch(train)  [18][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:59:53  time: 0.2180  data_time: 0.0080  memory: 3639  loss: 4.5743  loss_cls: 0.7201  loss_bbox: 2.3963  loss_obj: 1.4580
03/19 19:04:28 - mmengine - INFO - Epoch(train)  [18][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:59:43  time: 0.2206  data_time: 0.0081  memory: 3937  loss: 4.5664  loss_cls: 0.7094  loss_bbox: 2.3658  loss_obj: 1.4913
03/19 19:04:38 - mmengine - INFO - Epoch(train)  [18][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:59:29  time: 0.2051  data_time: 0.0080  memory: 2817  loss: 4.5594  loss_cls: 0.7055  loss_bbox: 2.3908  loss_obj: 1.4630
03/19 19:04:48 - mmengine - INFO - Epoch(train)  [18][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:59:16  time: 0.2092  data_time: 0.0080  memory: 3071  loss: 4.5641  loss_cls: 0.7182  loss_bbox: 2.3894  loss_obj: 1.4565
03/19 19:05:00 - mmengine - INFO - Epoch(train)  [18][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:59:09  time: 0.2315  data_time: 0.0081  memory: 3937  loss: 4.6046  loss_cls: 0.7169  loss_bbox: 2.3888  loss_obj: 1.4989
03/19 19:05:10 - mmengine - INFO - Epoch(train)  [18][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:58:53  time: 0.1980  data_time: 0.0081  memory: 3071  loss: 4.5499  loss_cls: 0.7127  loss_bbox: 2.4102  loss_obj: 1.4270
03/19 19:05:22 - mmengine - INFO - Epoch(train)  [18][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:58:47  time: 0.2398  data_time: 0.0080  memory: 3937  loss: 4.4882  loss_cls: 0.7051  loss_bbox: 2.3651  loss_obj: 1.4180
03/19 19:05:31 - mmengine - INFO - Epoch(train)  [18][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:58:30  time: 0.1905  data_time: 0.0080  memory: 2587  loss: 4.5317  loss_cls: 0.7189  loss_bbox: 2.4091  loss_obj: 1.4037
03/19 19:05:41 - mmengine - INFO - Epoch(train)  [18][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:58:13  time: 0.1921  data_time: 0.0082  memory: 3639  loss: 4.5948  loss_cls: 0.7185  loss_bbox: 2.4236  loss_obj: 1.4527
03/19 19:05:51 - mmengine - INFO - Epoch(train)  [18][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:59  time: 0.2014  data_time: 0.0080  memory: 3357  loss: 4.4852  loss_cls: 0.7027  loss_bbox: 2.3742  loss_obj: 1.4083
03/19 19:06:02 - mmengine - INFO - Epoch(train)  [18][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:48  time: 0.2206  data_time: 0.0080  memory: 3937  loss: 4.5591  loss_cls: 0.7001  loss_bbox: 2.3922  loss_obj: 1.4668
03/19 19:06:14 - mmengine - INFO - Epoch(train)  [18][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:41  time: 0.2349  data_time: 0.0081  memory: 3937  loss: 4.5736  loss_cls: 0.7135  loss_bbox: 2.3969  loss_obj: 1.4632
03/19 19:06:25 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:06:25 - mmengine - INFO - Epoch(train)  [18][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:34  time: 0.2309  data_time: 0.0079  memory: 3937  loss: 4.6070  loss_cls: 0.7136  loss_bbox: 2.4036  loss_obj: 1.4899
03/19 19:06:25 - mmengine - INFO - Saving checkpoint at 18 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:06:31 - mmengine - INFO - Epoch(val)  [18][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0071  memory: 527  
03/19 19:06:34 - mmengine - INFO - Epoch(val)  [18][100/250]    eta: 0:00:09  time: 0.0615  data_time: 0.0090  memory: 527  
03/19 19:06:37 - mmengine - INFO - Epoch(val)  [18][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 19:06:40 - mmengine - INFO - Epoch(val)  [18][200/250]    eta: 0:00:02  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 19:06:42 - mmengine - INFO - Epoch(val)  [18][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 19:06:44 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.47s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.22s).
Accumulating evaluation results...
DONE (t=2.73s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.195
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.466
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.128
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.116
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.258
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.368
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.305
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.305
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.305
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.237
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.368
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.456
03/19 19:06:57 - mmengine - INFO - bbox_mAP_copypaste: 0.195 0.466 0.128 0.116 0.258 0.368
03/19 19:06:57 - mmengine - INFO - Epoch(val) [18][250/250]    coco/bbox_mAP: 0.1950  coco/bbox_mAP_50: 0.4660  coco/bbox_mAP_75: 0.1280  coco/bbox_mAP_s: 0.1160  coco/bbox_mAP_m: 0.2580  coco/bbox_mAP_l: 0.3680  data_time: 0.0071  time: 0.0592
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:07:09 - mmengine - INFO - Epoch(train)  [19][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:26  time: 0.2303  data_time: 0.0185  memory: 3937  loss: 4.5448  loss_cls: 0.7105  loss_bbox: 2.3826  loss_obj: 1.4517
03/19 19:07:19 - mmengine - INFO - Epoch(train)  [19][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:57:11  time: 0.2028  data_time: 0.0079  memory: 2817  loss: 4.5041  loss_cls: 0.7064  loss_bbox: 2.3841  loss_obj: 1.4136
03/19 19:07:28 - mmengine - INFO - Epoch(train)  [19][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:54  time: 0.1913  data_time: 0.0082  memory: 3357  loss: 4.5802  loss_cls: 0.7225  loss_bbox: 2.4220  loss_obj: 1.4356
03/19 19:07:41 - mmengine - INFO - Epoch(train)  [19][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:50  time: 0.2471  data_time: 0.0080  memory: 3937  loss: 4.4983  loss_cls: 0.6928  loss_bbox: 2.3363  loss_obj: 1.4693
03/19 19:07:52 - mmengine - INFO - Epoch(train)  [19][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:40  time: 0.2190  data_time: 0.0081  memory: 3639  loss: 4.5018  loss_cls: 0.7026  loss_bbox: 2.3650  loss_obj: 1.4342
03/19 19:08:03 - mmengine - INFO - Epoch(train)  [19][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:28  time: 0.2153  data_time: 0.0081  memory: 3639  loss: 4.5162  loss_cls: 0.7139  loss_bbox: 2.3724  loss_obj: 1.4299
03/19 19:08:13 - mmengine - INFO - Epoch(train)  [19][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:17  time: 0.2161  data_time: 0.0080  memory: 3937  loss: 4.4669  loss_cls: 0.6910  loss_bbox: 2.3625  loss_obj: 1.4134
03/19 19:08:26 - mmengine - INFO - Epoch(train)  [19][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:13  time: 0.2497  data_time: 0.0080  memory: 3937  loss: 4.4803  loss_cls: 0.6991  loss_bbox: 2.3305  loss_obj: 1.4507
03/19 19:08:36 - mmengine - INFO - Epoch(train)  [19][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:56:00  time: 0.2093  data_time: 0.0080  memory: 3071  loss: 4.5490  loss_cls: 0.7092  loss_bbox: 2.3875  loss_obj: 1.4523
03/19 19:08:48 - mmengine - INFO - Epoch(train)  [19][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:55:55  time: 0.2421  data_time: 0.0079  memory: 3937  loss: 4.5028  loss_cls: 0.6969  loss_bbox: 2.3776  loss_obj: 1.4284
03/19 19:09:00 - mmengine - INFO - Epoch(train)  [19][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:55:47  time: 0.2305  data_time: 0.0081  memory: 3937  loss: 4.5176  loss_cls: 0.6876  loss_bbox: 2.3893  loss_obj: 1.4407
03/19 19:09:11 - mmengine - INFO - Epoch(train)  [19][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:55:35  time: 0.2130  data_time: 0.0081  memory: 3937  loss: 4.5670  loss_cls: 0.7051  loss_bbox: 2.3863  loss_obj: 1.4757
03/19 19:09:22 - mmengine - INFO - Epoch(train)  [19][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:55:28  time: 0.2352  data_time: 0.0079  memory: 3639  loss: 4.5796  loss_cls: 0.7134  loss_bbox: 2.3838  loss_obj: 1.4825
03/19 19:09:32 - mmengine - INFO - Epoch(train)  [19][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:55:12  time: 0.1958  data_time: 0.0081  memory: 2817  loss: 4.5403  loss_cls: 0.7228  loss_bbox: 2.4103  loss_obj: 1.4073
03/19 19:09:42 - mmengine - INFO - Epoch(train)  [19][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:58  time: 0.2042  data_time: 0.0081  memory: 3357  loss: 4.4268  loss_cls: 0.7035  loss_bbox: 2.3672  loss_obj: 1.3561
03/19 19:09:55 - mmengine - INFO - Epoch(train)  [19][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:53  time: 0.2433  data_time: 0.0080  memory: 3937  loss: 4.5746  loss_cls: 0.7159  loss_bbox: 2.3697  loss_obj: 1.4890
03/19 19:10:06 - mmengine - INFO - Epoch(train)  [19][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:45  time: 0.2351  data_time: 0.0080  memory: 3937  loss: 4.5370  loss_cls: 0.7076  loss_bbox: 2.3868  loss_obj: 1.4427
03/19 19:10:18 - mmengine - INFO - Epoch(train)  [19][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:36  time: 0.2230  data_time: 0.0080  memory: 3937  loss: 4.5119  loss_cls: 0.7004  loss_bbox: 2.3720  loss_obj: 1.4396
03/19 19:10:28 - mmengine - INFO - Epoch(train)  [19][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:22  time: 0.2068  data_time: 0.0081  memory: 3639  loss: 4.4846  loss_cls: 0.7043  loss_bbox: 2.3760  loss_obj: 1.4043
03/19 19:10:37 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:10:37 - mmengine - INFO - Epoch(train)  [19][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:54:05  time: 0.1898  data_time: 0.0081  memory: 2587  loss: 4.5328  loss_cls: 0.7192  loss_bbox: 2.4020  loss_obj: 1.4116
03/19 19:10:37 - mmengine - INFO - Saving checkpoint at 19 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:10:43 - mmengine - INFO - Epoch(val)  [19][ 50/250]    eta: 0:00:11  time: 0.0584  data_time: 0.0071  memory: 527  
03/19 19:10:46 - mmengine - INFO - Epoch(val)  [19][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 19:10:49 - mmengine - INFO - Epoch(val)  [19][150/250]    eta: 0:00:05  time: 0.0591  data_time: 0.0064  memory: 527  
03/19 19:10:51 - mmengine - INFO - Epoch(val)  [19][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 19:10:54 - mmengine - INFO - Epoch(val)  [19][250/250]    eta: 0:00:00  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 19:10:56 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.30s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.24s).
Accumulating evaluation results...
DONE (t=2.66s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.197
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.470
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.129
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.118
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.258
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.388
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.306
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.306
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.306
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.240
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.468
03/19 19:11:09 - mmengine - INFO - bbox_mAP_copypaste: 0.197 0.470 0.129 0.118 0.258 0.388
03/19 19:11:09 - mmengine - INFO - Epoch(val) [19][250/250]    coco/bbox_mAP: 0.1970  coco/bbox_mAP_50: 0.4700  coco/bbox_mAP_75: 0.1290  coco/bbox_mAP_s: 0.1180  coco/bbox_mAP_m: 0.2580  coco/bbox_mAP_l: 0.3880  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:11:19 - mmengine - INFO - Epoch(train)  [20][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:53:51  time: 0.2020  data_time: 0.0191  memory: 2587  loss: 4.5421  loss_cls: 0.7135  loss_bbox: 2.4160  loss_obj: 1.4125
03/19 19:11:30 - mmengine - INFO - Epoch(train)  [20][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:53:39  time: 0.2095  data_time: 0.0081  memory: 3639  loss: 4.5201  loss_cls: 0.7110  loss_bbox: 2.3834  loss_obj: 1.4257
03/19 19:11:40 - mmengine - INFO - Epoch(train)  [20][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:53:28  time: 0.2176  data_time: 0.0081  memory: 3639  loss: 4.6007  loss_cls: 0.7184  loss_bbox: 2.3950  loss_obj: 1.4873
03/19 19:11:51 - mmengine - INFO - Epoch(train)  [20][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:53:16  time: 0.2122  data_time: 0.0080  memory: 3639  loss: 4.5202  loss_cls: 0.7021  loss_bbox: 2.3810  loss_obj: 1.4372
03/19 19:12:03 - mmengine - INFO - Epoch(train)  [20][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:53:08  time: 0.2325  data_time: 0.0080  memory: 3639  loss: 4.5608  loss_cls: 0.7107  loss_bbox: 2.3661  loss_obj: 1.4840
03/19 19:12:13 - mmengine - INFO - Epoch(train)  [20][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:54  time: 0.2062  data_time: 0.0082  memory: 3357  loss: 4.5468  loss_cls: 0.7070  loss_bbox: 2.4127  loss_obj: 1.4271
03/19 19:12:23 - mmengine - INFO - Epoch(train)  [20][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:41  time: 0.2045  data_time: 0.0080  memory: 2817  loss: 4.5335  loss_cls: 0.7005  loss_bbox: 2.3844  loss_obj: 1.4485
03/19 19:12:34 - mmengine - INFO - Epoch(train)  [20][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:31  time: 0.2220  data_time: 0.0081  memory: 3937  loss: 4.4590  loss_cls: 0.6977  loss_bbox: 2.3504  loss_obj: 1.4109
03/19 19:12:47 - mmengine - INFO - Epoch(train)  [20][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:26  time: 0.2458  data_time: 0.0081  memory: 3937  loss: 4.4541  loss_cls: 0.6853  loss_bbox: 2.3490  loss_obj: 1.4198
03/19 19:12:57 - mmengine - INFO - Epoch(train)  [20][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:13  time: 0.2078  data_time: 0.0079  memory: 3071  loss: 4.5646  loss_cls: 0.7103  loss_bbox: 2.4073  loss_obj: 1.4470
03/19 19:13:08 - mmengine - INFO - Epoch(train)  [20][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:52:00  time: 0.2092  data_time: 0.0080  memory: 3071  loss: 4.5353  loss_cls: 0.7127  loss_bbox: 2.4040  loss_obj: 1.4185
03/19 19:13:19 - mmengine - INFO - Epoch(train)  [20][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:51:50  time: 0.2241  data_time: 0.0079  memory: 3937  loss: 4.4807  loss_cls: 0.7078  loss_bbox: 2.3688  loss_obj: 1.4041
03/19 19:13:29 - mmengine - INFO - Epoch(train)  [20][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:51:36  time: 0.2003  data_time: 0.0080  memory: 3071  loss: 4.5567  loss_cls: 0.7143  loss_bbox: 2.3968  loss_obj: 1.4457
03/19 19:13:40 - mmengine - INFO - Epoch(train)  [20][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:51:25  time: 0.2172  data_time: 0.0080  memory: 3357  loss: 4.4939  loss_cls: 0.7131  loss_bbox: 2.3544  loss_obj: 1.4265
03/19 19:13:51 - mmengine - INFO - Epoch(train)  [20][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:51:17  time: 0.2325  data_time: 0.0079  memory: 3639  loss: 4.5429  loss_cls: 0.7005  loss_bbox: 2.3814  loss_obj: 1.4609
03/19 19:14:02 - mmengine - INFO - Epoch(train)  [20][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:51:06  time: 0.2177  data_time: 0.0080  memory: 3639  loss: 4.5387  loss_cls: 0.7019  loss_bbox: 2.3828  loss_obj: 1.4541
03/19 19:14:13 - mmengine - INFO - Epoch(train)  [20][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:55  time: 0.2158  data_time: 0.0079  memory: 3639  loss: 4.5239  loss_cls: 0.7020  loss_bbox: 2.3822  loss_obj: 1.4398
03/19 19:14:25 - mmengine - INFO - Epoch(train)  [20][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:50  time: 0.2484  data_time: 0.0079  memory: 3937  loss: 4.5021  loss_cls: 0.6920  loss_bbox: 2.3525  loss_obj: 1.4575
03/19 19:14:38 - mmengine - INFO - Epoch(train)  [20][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:44  time: 0.2427  data_time: 0.0079  memory: 3937  loss: 4.5239  loss_cls: 0.7024  loss_bbox: 2.3612  loss_obj: 1.4603
03/19 19:14:48 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:14:48 - mmengine - INFO - Epoch(train)  [20][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:31  time: 0.2062  data_time: 0.0079  memory: 3357  loss: 4.5234  loss_cls: 0.7076  loss_bbox: 2.4060  loss_obj: 1.4099
03/19 19:14:48 - mmengine - INFO - Saving checkpoint at 20 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:14:53 - mmengine - INFO - Epoch(val)  [20][ 50/250]    eta: 0:00:11  time: 0.0591  data_time: 0.0071  memory: 527  
03/19 19:14:56 - mmengine - INFO - Epoch(val)  [20][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 19:14:59 - mmengine - INFO - Epoch(val)  [20][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 19:15:02 - mmengine - INFO - Epoch(val)  [20][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 19:15:05 - mmengine - INFO - Epoch(val)  [20][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 19:15:07 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.47s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.84s).
Accumulating evaluation results...
DONE (t=2.61s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.197
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.473
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.127
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.117
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.261
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.403
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.241
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.374
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.466
03/19 19:15:19 - mmengine - INFO - bbox_mAP_copypaste: 0.197 0.473 0.127 0.117 0.261 0.403
03/19 19:15:19 - mmengine - INFO - Epoch(val) [20][250/250]    coco/bbox_mAP: 0.1970  coco/bbox_mAP_50: 0.4730  coco/bbox_mAP_75: 0.1270  coco/bbox_mAP_s: 0.1170  coco/bbox_mAP_m: 0.2610  coco/bbox_mAP_l: 0.4030  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:15:32 - mmengine - INFO - Epoch(train)  [21][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:28  time: 0.2579  data_time: 0.0184  memory: 3937  loss: 4.5500  loss_cls: 0.7017  loss_bbox: 2.3840  loss_obj: 1.4643
03/19 19:15:43 - mmengine - INFO - Epoch(train)  [21][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:19  time: 0.2256  data_time: 0.0080  memory: 3639  loss: 4.4462  loss_cls: 0.6960  loss_bbox: 2.3451  loss_obj: 1.4050
03/19 19:15:54 - mmengine - INFO - Epoch(train)  [21][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:50:05  time: 0.2042  data_time: 0.0080  memory: 3639  loss: 4.5010  loss_cls: 0.6971  loss_bbox: 2.3814  loss_obj: 1.4225
03/19 19:16:05 - mmengine - INFO - Epoch(train)  [21][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:49:55  time: 0.2243  data_time: 0.0081  memory: 3937  loss: 4.5293  loss_cls: 0.7021  loss_bbox: 2.3870  loss_obj: 1.4402
03/19 19:16:15 - mmengine - INFO - Epoch(train)  [21][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:49:41  time: 0.2003  data_time: 0.0080  memory: 3357  loss: 4.5279  loss_cls: 0.7065  loss_bbox: 2.4103  loss_obj: 1.4112
03/19 19:16:26 - mmengine - INFO - Epoch(train)  [21][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:49:29  time: 0.2142  data_time: 0.0079  memory: 3639  loss: 4.5262  loss_cls: 0.7102  loss_bbox: 2.3974  loss_obj: 1.4185
03/19 19:16:36 - mmengine - INFO - Epoch(train)  [21][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:49:17  time: 0.2087  data_time: 0.0082  memory: 3937  loss: 4.6093  loss_cls: 0.7171  loss_bbox: 2.4289  loss_obj: 1.4633
03/19 19:16:46 - mmengine - INFO - Epoch(train)  [21][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:49:03  time: 0.2051  data_time: 0.0080  memory: 2817  loss: 4.5525  loss_cls: 0.7142  loss_bbox: 2.3872  loss_obj: 1.4512
03/19 19:16:57 - mmengine - INFO - Epoch(train)  [21][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:48:51  time: 0.2082  data_time: 0.0081  memory: 3937  loss: 4.5269  loss_cls: 0.7111  loss_bbox: 2.3858  loss_obj: 1.4300
03/19 19:17:07 - mmengine - INFO - Epoch(train)  [21][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:48:37  time: 0.2065  data_time: 0.0080  memory: 3071  loss: 4.4856  loss_cls: 0.7039  loss_bbox: 2.3586  loss_obj: 1.4231
03/19 19:17:17 - mmengine - INFO - Epoch(train)  [21][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:48:22  time: 0.1921  data_time: 0.0082  memory: 2817  loss: 4.5377  loss_cls: 0.7167  loss_bbox: 2.4168  loss_obj: 1.4042
03/19 19:17:27 - mmengine - INFO - Epoch(train)  [21][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:48:08  time: 0.2048  data_time: 0.0080  memory: 3071  loss: 4.5004  loss_cls: 0.7099  loss_bbox: 2.3866  loss_obj: 1.4039
03/19 19:17:37 - mmengine - INFO - Epoch(train)  [21][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:55  time: 0.2075  data_time: 0.0083  memory: 3937  loss: 4.5239  loss_cls: 0.7122  loss_bbox: 2.3896  loss_obj: 1.4221
03/19 19:17:49 - mmengine - INFO - Epoch(train)  [21][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:49  time: 0.2436  data_time: 0.0080  memory: 3937  loss: 4.5265  loss_cls: 0.6930  loss_bbox: 2.3696  loss_obj: 1.4639
03/19 19:18:01 - mmengine - INFO - Epoch(train)  [21][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:40  time: 0.2251  data_time: 0.0080  memory: 3639  loss: 4.4757  loss_cls: 0.6894  loss_bbox: 2.3482  loss_obj: 1.4380
03/19 19:18:12 - mmengine - INFO - Epoch(train)  [21][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:31  time: 0.2257  data_time: 0.0080  memory: 3937  loss: 4.5944  loss_cls: 0.7131  loss_bbox: 2.3784  loss_obj: 1.5029
03/19 19:18:22 - mmengine - INFO - Epoch(train)  [21][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:16  time: 0.1963  data_time: 0.0083  memory: 3357  loss: 4.5415  loss_cls: 0.7127  loss_bbox: 2.3957  loss_obj: 1.4330
03/19 19:18:32 - mmengine - INFO - Epoch(train)  [21][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:47:03  time: 0.2067  data_time: 0.0081  memory: 3937  loss: 4.5621  loss_cls: 0.7151  loss_bbox: 2.3862  loss_obj: 1.4609
03/19 19:18:43 - mmengine - INFO - Epoch(train)  [21][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:46:50  time: 0.2102  data_time: 0.0082  memory: 3639  loss: 4.4898  loss_cls: 0.7035  loss_bbox: 2.3819  loss_obj: 1.4044
03/19 19:18:53 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:18:53 - mmengine - INFO - Epoch(train)  [21][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:46:38  time: 0.2123  data_time: 0.0081  memory: 3357  loss: 4.4858  loss_cls: 0.7058  loss_bbox: 2.3646  loss_obj: 1.4154
03/19 19:18:53 - mmengine - INFO - Saving checkpoint at 21 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:18:59 - mmengine - INFO - Epoch(val)  [21][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 19:19:02 - mmengine - INFO - Epoch(val)  [21][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 19:19:05 - mmengine - INFO - Epoch(val)  [21][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 19:19:07 - mmengine - INFO - Epoch(val)  [21][200/250]    eta: 0:00:02  time: 0.0595  data_time: 0.0065  memory: 527  
03/19 19:19:10 - mmengine - INFO - Epoch(val)  [21][250/250]    eta: 0:00:00  time: 0.0578  data_time: 0.0066  memory: 527  
03/19 19:19:12 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.30s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.99s).
Accumulating evaluation results...
DONE (t=2.57s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.198
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.477
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.127
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.118
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.260
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.408
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.307
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.307
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.307
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.239
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.372
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.465
03/19 19:19:25 - mmengine - INFO - bbox_mAP_copypaste: 0.198 0.477 0.127 0.118 0.260 0.408
03/19 19:19:25 - mmengine - INFO - Epoch(val) [21][250/250]    coco/bbox_mAP: 0.1980  coco/bbox_mAP_50: 0.4770  coco/bbox_mAP_75: 0.1270  coco/bbox_mAP_s: 0.1180  coco/bbox_mAP_m: 0.2600  coco/bbox_mAP_l: 0.4080  data_time: 0.0066  time: 0.0588
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:19:37 - mmengine - INFO - Epoch(train)  [22][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:46:33  time: 0.2480  data_time: 0.0186  memory: 3937  loss: 4.6075  loss_cls: 0.7110  loss_bbox: 2.3995  loss_obj: 1.4971
03/19 19:19:48 - mmengine - INFO - Epoch(train)  [22][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:46:22  time: 0.2166  data_time: 0.0082  memory: 3937  loss: 4.5871  loss_cls: 0.7132  loss_bbox: 2.3917  loss_obj: 1.4822
03/19 19:19:59 - mmengine - INFO - Epoch(train)  [22][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:46:12  time: 0.2219  data_time: 0.0080  memory: 3937  loss: 4.5109  loss_cls: 0.7057  loss_bbox: 2.3695  loss_obj: 1.4358
03/19 19:20:08 - mmengine - INFO - Epoch(train)  [22][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:45:54  time: 0.1780  data_time: 0.0084  memory: 2587  loss: 4.5730  loss_cls: 0.7234  loss_bbox: 2.4235  loss_obj: 1.4262
03/19 19:20:19 - mmengine - INFO - Epoch(train)  [22][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:45:45  time: 0.2294  data_time: 0.0081  memory: 3937  loss: 4.4821  loss_cls: 0.6991  loss_bbox: 2.3694  loss_obj: 1.4136
03/19 19:20:30 - mmengine - INFO - Epoch(train)  [22][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:45:33  time: 0.2142  data_time: 0.0082  memory: 3071  loss: 4.4819  loss_cls: 0.7073  loss_bbox: 2.3879  loss_obj: 1.3867
03/19 19:20:41 - mmengine - INFO - Epoch(train)  [22][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:45:24  time: 0.2247  data_time: 0.0080  memory: 3937  loss: 4.4715  loss_cls: 0.7060  loss_bbox: 2.3500  loss_obj: 1.4154
03/19 19:20:52 - mmengine - INFO - Epoch(train)  [22][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:45:11  time: 0.2054  data_time: 0.0080  memory: 3357  loss: 4.5219  loss_cls: 0.7127  loss_bbox: 2.3872  loss_obj: 1.4220
03/19 19:21:02 - mmengine - INFO - Epoch(train)  [22][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:44:57  time: 0.2031  data_time: 0.0081  memory: 3071  loss: 4.5269  loss_cls: 0.7182  loss_bbox: 2.3909  loss_obj: 1.4178
03/19 19:21:12 - mmengine - INFO - Epoch(train)  [22][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:44:45  time: 0.2133  data_time: 0.0081  memory: 3639  loss: 4.4826  loss_cls: 0.7011  loss_bbox: 2.3499  loss_obj: 1.4316
03/19 19:21:23 - mmengine - INFO - Epoch(train)  [22][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:44:33  time: 0.2094  data_time: 0.0082  memory: 3639  loss: 4.4766  loss_cls: 0.7003  loss_bbox: 2.3799  loss_obj: 1.3964
03/19 19:21:33 - mmengine - INFO - Epoch(train)  [22][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:44:21  time: 0.2091  data_time: 0.0083  memory: 3071  loss: 4.4642  loss_cls: 0.7118  loss_bbox: 2.3663  loss_obj: 1.3862
03/19 19:21:44 - mmengine - INFO - Epoch(train)  [22][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:44:09  time: 0.2114  data_time: 0.0081  memory: 3639  loss: 4.4881  loss_cls: 0.7049  loss_bbox: 2.3772  loss_obj: 1.4060
03/19 19:21:54 - mmengine - INFO - Epoch(train)  [22][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:56  time: 0.2085  data_time: 0.0079  memory: 3357  loss: 4.4906  loss_cls: 0.7011  loss_bbox: 2.3975  loss_obj: 1.3920
03/19 19:22:06 - mmengine - INFO - Epoch(train)  [22][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:47  time: 0.2297  data_time: 0.0081  memory: 3937  loss: 4.5683  loss_cls: 0.7100  loss_bbox: 2.3752  loss_obj: 1.4832
03/19 19:22:18 - mmengine - INFO - Epoch(train)  [22][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:40  time: 0.2373  data_time: 0.0080  memory: 3941  loss: 4.4601  loss_cls: 0.6987  loss_bbox: 2.3477  loss_obj: 1.4137
03/19 19:22:29 - mmengine - INFO - Epoch(train)  [22][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:30  time: 0.2235  data_time: 0.0080  memory: 3937  loss: 4.5150  loss_cls: 0.7030  loss_bbox: 2.3829  loss_obj: 1.4290
03/19 19:22:39 - mmengine - INFO - Epoch(train)  [22][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:17  time: 0.2067  data_time: 0.0079  memory: 2817  loss: 4.5474  loss_cls: 0.7140  loss_bbox: 2.3880  loss_obj: 1.4454
03/19 19:22:50 - mmengine - INFO - Epoch(train)  [22][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:43:06  time: 0.2137  data_time: 0.0079  memory: 3639  loss: 4.3890  loss_cls: 0.6894  loss_bbox: 2.3172  loss_obj: 1.3824
03/19 19:23:00 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:23:00 - mmengine - INFO - Epoch(train)  [22][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:53  time: 0.2042  data_time: 0.0079  memory: 3071  loss: 4.5349  loss_cls: 0.7018  loss_bbox: 2.3884  loss_obj: 1.4448
03/19 19:23:00 - mmengine - INFO - Saving checkpoint at 22 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:23:06 - mmengine - INFO - Epoch(val)  [22][ 50/250]    eta: 0:00:11  time: 0.0597  data_time: 0.0071  memory: 527  
03/19 19:23:09 - mmengine - INFO - Epoch(val)  [22][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0066  memory: 527  
03/19 19:23:11 - mmengine - INFO - Epoch(val)  [22][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 19:23:14 - mmengine - INFO - Epoch(val)  [22][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 19:23:17 - mmengine - INFO - Epoch(val)  [22][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0064  memory: 527  
03/19 19:23:19 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.30s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.87s).
Accumulating evaluation results...
DONE (t=2.52s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.199
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.477
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.128
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.119
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.263
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.396
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.308
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.308
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.308
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.240
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.376
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.461
03/19 19:23:31 - mmengine - INFO - bbox_mAP_copypaste: 0.199 0.477 0.128 0.119 0.263 0.396
03/19 19:23:31 - mmengine - INFO - Epoch(val) [22][250/250]    coco/bbox_mAP: 0.1990  coco/bbox_mAP_50: 0.4770  coco/bbox_mAP_75: 0.1280  coco/bbox_mAP_s: 0.1190  coco/bbox_mAP_m: 0.2630  coco/bbox_mAP_l: 0.3960  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:23:43 - mmengine - INFO - Epoch(train)  [23][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:44  time: 0.2324  data_time: 0.0189  memory: 3357  loss: 4.4948  loss_cls: 0.7005  loss_bbox: 2.3562  loss_obj: 1.4381
03/19 19:23:54 - mmengine - INFO - Epoch(train)  [23][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:35  time: 0.2243  data_time: 0.0081  memory: 3357  loss: 4.4528  loss_cls: 0.6867  loss_bbox: 2.3632  loss_obj: 1.4029
03/19 19:24:05 - mmengine - INFO - Epoch(train)  [23][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:23  time: 0.2159  data_time: 0.0080  memory: 3639  loss: 4.3993  loss_cls: 0.6887  loss_bbox: 2.3490  loss_obj: 1.3616
03/19 19:24:16 - mmengine - INFO - Epoch(train)  [23][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:13  time: 0.2206  data_time: 0.0080  memory: 3937  loss: 4.5104  loss_cls: 0.7031  loss_bbox: 2.3737  loss_obj: 1.4336
03/19 19:24:27 - mmengine - INFO - Epoch(train)  [23][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:42:01  time: 0.2115  data_time: 0.0080  memory: 3357  loss: 4.5136  loss_cls: 0.7058  loss_bbox: 2.3757  loss_obj: 1.4322
03/19 19:24:37 - mmengine - INFO - Epoch(train)  [23][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:41:48  time: 0.2059  data_time: 0.0081  memory: 3357  loss: 4.4861  loss_cls: 0.6967  loss_bbox: 2.3865  loss_obj: 1.4030
03/19 19:24:47 - mmengine - INFO - Epoch(train)  [23][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:41:34  time: 0.1989  data_time: 0.0080  memory: 2817  loss: 4.5643  loss_cls: 0.7087  loss_bbox: 2.4189  loss_obj: 1.4367
03/19 19:24:58 - mmengine - INFO - Epoch(train)  [23][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:41:23  time: 0.2166  data_time: 0.0079  memory: 3357  loss: 4.5121  loss_cls: 0.6938  loss_bbox: 2.3742  loss_obj: 1.4440
03/19 19:25:09 - mmengine - INFO - Epoch(train)  [23][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:41:12  time: 0.2189  data_time: 0.0079  memory: 3639  loss: 4.5049  loss_cls: 0.6978  loss_bbox: 2.3677  loss_obj: 1.4394
03/19 19:25:21 - mmengine - INFO - Epoch(train)  [23][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:41:06  time: 0.2457  data_time: 0.0080  memory: 3937  loss: 4.4989  loss_cls: 0.6894  loss_bbox: 2.3533  loss_obj: 1.4562
03/19 19:25:32 - mmengine - INFO - Epoch(train)  [23][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:40:57  time: 0.2259  data_time: 0.0081  memory: 3357  loss: 4.5579  loss_cls: 0.7153  loss_bbox: 2.3907  loss_obj: 1.4520
03/19 19:25:42 - mmengine - INFO - Epoch(train)  [23][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:40:43  time: 0.1981  data_time: 0.0080  memory: 2817  loss: 4.4434  loss_cls: 0.7020  loss_bbox: 2.3603  loss_obj: 1.3811
03/19 19:25:52 - mmengine - INFO - Epoch(train)  [23][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:40:29  time: 0.2013  data_time: 0.0081  memory: 2817  loss: 4.4528  loss_cls: 0.6986  loss_bbox: 2.3639  loss_obj: 1.3904
03/19 19:26:03 - mmengine - INFO - Epoch(train)  [23][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:40:19  time: 0.2239  data_time: 0.0081  memory: 3937  loss: 4.4666  loss_cls: 0.6950  loss_bbox: 2.3560  loss_obj: 1.4156
03/19 19:26:14 - mmengine - INFO - Epoch(train)  [23][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:40:06  time: 0.2035  data_time: 0.0080  memory: 3639  loss: 4.4490  loss_cls: 0.7122  loss_bbox: 2.3523  loss_obj: 1.3846
03/19 19:26:25 - mmengine - INFO - Epoch(train)  [23][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:55  time: 0.2183  data_time: 0.0081  memory: 3937  loss: 4.4989  loss_cls: 0.7004  loss_bbox: 2.3867  loss_obj: 1.4117
03/19 19:26:35 - mmengine - INFO - Epoch(train)  [23][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:43  time: 0.2100  data_time: 0.0082  memory: 3357  loss: 4.4481  loss_cls: 0.6896  loss_bbox: 2.3483  loss_obj: 1.4101
03/19 19:26:45 - mmengine - INFO - Epoch(train)  [23][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:30  time: 0.2064  data_time: 0.0080  memory: 3071  loss: 4.4719  loss_cls: 0.7021  loss_bbox: 2.3614  loss_obj: 1.4084
03/19 19:26:57 - mmengine - INFO - Epoch(train)  [23][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:21  time: 0.2250  data_time: 0.0081  memory: 3937  loss: 4.5165  loss_cls: 0.7113  loss_bbox: 2.3823  loss_obj: 1.4230
03/19 19:27:07 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:27:07 - mmengine - INFO - Epoch(train)  [23][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:09  time: 0.2154  data_time: 0.0080  memory: 3639  loss: 4.5341  loss_cls: 0.6935  loss_bbox: 2.3976  loss_obj: 1.4430
03/19 19:27:07 - mmengine - INFO - Saving checkpoint at 23 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:27:13 - mmengine - INFO - Epoch(val)  [23][ 50/250]    eta: 0:00:12  time: 0.0625  data_time: 0.0100  memory: 527  
03/19 19:27:16 - mmengine - INFO - Epoch(val)  [23][100/250]    eta: 0:00:09  time: 0.0589  data_time: 0.0066  memory: 527  
03/19 19:27:19 - mmengine - INFO - Epoch(val)  [23][150/250]    eta: 0:00:06  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 19:27:21 - mmengine - INFO - Epoch(val)  [23][200/250]    eta: 0:00:02  time: 0.0579  data_time: 0.0064  memory: 527  
03/19 19:27:24 - mmengine - INFO - Epoch(val)  [23][250/250]    eta: 0:00:00  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 19:27:26 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.29s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.72s).
Accumulating evaluation results...
DONE (t=2.47s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.201
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.483
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.133
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.120
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.265
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.241
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.378
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.458
03/19 19:27:38 - mmengine - INFO - bbox_mAP_copypaste: 0.201 0.483 0.133 0.120 0.265 0.392
03/19 19:27:38 - mmengine - INFO - Epoch(val) [23][250/250]    coco/bbox_mAP: 0.2010  coco/bbox_mAP_50: 0.4830  coco/bbox_mAP_75: 0.1330  coco/bbox_mAP_s: 0.1200  coco/bbox_mAP_m: 0.2650  coco/bbox_mAP_l: 0.3920  data_time: 0.0072  time: 0.0592
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:27:50 - mmengine - INFO - Epoch(train)  [24][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:39:01  time: 0.2331  data_time: 0.0191  memory: 3937  loss: 4.5625  loss_cls: 0.7023  loss_bbox: 2.3965  loss_obj: 1.4638
03/19 19:28:01 - mmengine - INFO - Epoch(train)  [24][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:38:50  time: 0.2174  data_time: 0.0082  memory: 3639  loss: 4.4628  loss_cls: 0.7029  loss_bbox: 2.3491  loss_obj: 1.4108
03/19 19:28:11 - mmengine - INFO - Epoch(train)  [24][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:38:37  time: 0.2045  data_time: 0.0080  memory: 3639  loss: 4.5023  loss_cls: 0.7041  loss_bbox: 2.3941  loss_obj: 1.4042
03/19 19:28:21 - mmengine - INFO - Epoch(train)  [24][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:38:25  time: 0.2131  data_time: 0.0081  memory: 3357  loss: 4.5067  loss_cls: 0.7057  loss_bbox: 2.3777  loss_obj: 1.4233
03/19 19:28:32 - mmengine - INFO - Epoch(train)  [24][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:38:14  time: 0.2149  data_time: 0.0079  memory: 3937  loss: 4.4605  loss_cls: 0.7000  loss_bbox: 2.3646  loss_obj: 1.3958
03/19 19:28:42 - mmengine - INFO - Epoch(train)  [24][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:38:00  time: 0.1953  data_time: 0.0082  memory: 3639  loss: 4.5436  loss_cls: 0.7133  loss_bbox: 2.3960  loss_obj: 1.4343
03/19 19:28:52 - mmengine - INFO - Epoch(train)  [24][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:37:47  time: 0.2068  data_time: 0.0080  memory: 3357  loss: 4.3987  loss_cls: 0.6942  loss_bbox: 2.3410  loss_obj: 1.3635
03/19 19:29:04 - mmengine - INFO - Epoch(train)  [24][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:37:37  time: 0.2235  data_time: 0.0081  memory: 3937  loss: 4.5581  loss_cls: 0.7101  loss_bbox: 2.3930  loss_obj: 1.4550
03/19 19:29:14 - mmengine - INFO - Epoch(train)  [24][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:37:24  time: 0.2063  data_time: 0.0082  memory: 3639  loss: 4.4861  loss_cls: 0.6951  loss_bbox: 2.3652  loss_obj: 1.4259
03/19 19:29:25 - mmengine - INFO - Epoch(train)  [24][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:37:13  time: 0.2158  data_time: 0.0080  memory: 3937  loss: 4.6236  loss_cls: 0.7191  loss_bbox: 2.4202  loss_obj: 1.4843
03/19 19:29:36 - mmengine - INFO - Epoch(train)  [24][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:37:02  time: 0.2171  data_time: 0.0081  memory: 3639  loss: 4.4667  loss_cls: 0.6857  loss_bbox: 2.3622  loss_obj: 1.4188
03/19 19:29:47 - mmengine - INFO - Epoch(train)  [24][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:36:53  time: 0.2291  data_time: 0.0081  memory: 3639  loss: 4.5424  loss_cls: 0.7118  loss_bbox: 2.3621  loss_obj: 1.4685
03/19 19:29:59 - mmengine - INFO - Epoch(train)  [24][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:36:45  time: 0.2310  data_time: 0.0080  memory: 3937  loss: 4.5282  loss_cls: 0.6948  loss_bbox: 2.3631  loss_obj: 1.4703
03/19 19:30:09 - mmengine - INFO - Epoch(train)  [24][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:36:33  time: 0.2150  data_time: 0.0079  memory: 3357  loss: 4.5121  loss_cls: 0.7006  loss_bbox: 2.3833  loss_obj: 1.4281
03/19 19:30:19 - mmengine - INFO - Epoch(train)  [24][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:36:20  time: 0.2018  data_time: 0.0082  memory: 3639  loss: 4.6115  loss_cls: 0.7189  loss_bbox: 2.4189  loss_obj: 1.4738
03/19 19:30:29 - mmengine - INFO - Epoch(train)  [24][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:36:06  time: 0.1989  data_time: 0.0081  memory: 3071  loss: 4.5450  loss_cls: 0.7079  loss_bbox: 2.3940  loss_obj: 1.4431
03/19 19:30:40 - mmengine - INFO - Epoch(train)  [24][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:35:53  time: 0.2028  data_time: 0.0083  memory: 3357  loss: 4.4743  loss_cls: 0.7010  loss_bbox: 2.3716  loss_obj: 1.4016
03/19 19:30:50 - mmengine - INFO - Epoch(train)  [24][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:35:40  time: 0.2025  data_time: 0.0080  memory: 3357  loss: 4.5561  loss_cls: 0.7152  loss_bbox: 2.4241  loss_obj: 1.4168
03/19 19:31:00 - mmengine - INFO - Epoch(train)  [24][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:35:27  time: 0.2060  data_time: 0.0080  memory: 3071  loss: 4.4821  loss_cls: 0.6994  loss_bbox: 2.3594  loss_obj: 1.4233
03/19 19:31:10 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:31:10 - mmengine - INFO - Epoch(train)  [24][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:35:14  time: 0.2058  data_time: 0.0078  memory: 3071  loss: 4.5410  loss_cls: 0.7082  loss_bbox: 2.3965  loss_obj: 1.4363
03/19 19:31:10 - mmengine - INFO - Saving checkpoint at 24 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:31:16 - mmengine - INFO - Epoch(val)  [24][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0071  memory: 527  
03/19 19:31:19 - mmengine - INFO - Epoch(val)  [24][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 19:31:22 - mmengine - INFO - Epoch(val)  [24][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0066  memory: 527  
03/19 19:31:25 - mmengine - INFO - Epoch(val)  [24][200/250]    eta: 0:00:02  time: 0.0594  data_time: 0.0066  memory: 527  
03/19 19:31:27 - mmengine - INFO - Epoch(val)  [24][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0064  memory: 527  
03/19 19:31:29 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.47s).
Accumulating evaluation results...
DONE (t=2.38s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.202
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.487
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.133
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.265
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.396
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.244
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.376
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.461
03/19 19:31:40 - mmengine - INFO - bbox_mAP_copypaste: 0.202 0.487 0.133 0.122 0.265 0.396
03/19 19:31:41 - mmengine - INFO - Epoch(val) [24][250/250]    coco/bbox_mAP: 0.2020  coco/bbox_mAP_50: 0.4870  coco/bbox_mAP_75: 0.1330  coco/bbox_mAP_s: 0.1220  coco/bbox_mAP_m: 0.2650  coco/bbox_mAP_l: 0.3960  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:31:53 - mmengine - INFO - Epoch(train)  [25][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:35:07  time: 0.2413  data_time: 0.0183  memory: 3937  loss: 4.4763  loss_cls: 0.6930  loss_bbox: 2.3438  loss_obj: 1.4395
03/19 19:32:04 - mmengine - INFO - Epoch(train)  [25][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:58  time: 0.2271  data_time: 0.0080  memory: 3937  loss: 4.4834  loss_cls: 0.7000  loss_bbox: 2.3592  loss_obj: 1.4243
03/19 19:32:15 - mmengine - INFO - Epoch(train)  [25][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:47  time: 0.2187  data_time: 0.0080  memory: 2817  loss: 4.4546  loss_cls: 0.6892  loss_bbox: 2.3580  loss_obj: 1.4074
03/19 19:32:26 - mmengine - INFO - Epoch(train)  [25][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:37  time: 0.2189  data_time: 0.0081  memory: 3937  loss: 4.5222  loss_cls: 0.7066  loss_bbox: 2.3856  loss_obj: 1.4300
03/19 19:32:37 - mmengine - INFO - Epoch(train)  [25][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:26  time: 0.2209  data_time: 0.0080  memory: 3937  loss: 4.5424  loss_cls: 0.7133  loss_bbox: 2.3920  loss_obj: 1.4370
03/19 19:32:48 - mmengine - INFO - Epoch(train)  [25][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:16  time: 0.2190  data_time: 0.0082  memory: 3639  loss: 4.5446  loss_cls: 0.7043  loss_bbox: 2.3822  loss_obj: 1.4581
03/19 19:32:58 - mmengine - INFO - Epoch(train)  [25][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:34:02  time: 0.1969  data_time: 0.0082  memory: 2817  loss: 4.5362  loss_cls: 0.7083  loss_bbox: 2.4086  loss_obj: 1.4193
03/19 19:33:09 - mmengine - INFO - Epoch(train)  [25][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:53  time: 0.2332  data_time: 0.0081  memory: 3937  loss: 4.4975  loss_cls: 0.6978  loss_bbox: 2.3647  loss_obj: 1.4350
03/19 19:33:20 - mmengine - INFO - Epoch(train)  [25][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:42  time: 0.2179  data_time: 0.0082  memory: 3639  loss: 4.4785  loss_cls: 0.6967  loss_bbox: 2.3771  loss_obj: 1.4048
03/19 19:33:32 - mmengine - INFO - Epoch(train)  [25][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:34  time: 0.2354  data_time: 0.0079  memory: 3937  loss: 4.5624  loss_cls: 0.7039  loss_bbox: 2.3701  loss_obj: 1.4884
03/19 19:33:42 - mmengine - INFO - Epoch(train)  [25][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:22  time: 0.2060  data_time: 0.0082  memory: 3357  loss: 4.4755  loss_cls: 0.6918  loss_bbox: 2.3594  loss_obj: 1.4243
03/19 19:33:55 - mmengine - INFO - Epoch(train)  [25][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:15  time: 0.2416  data_time: 0.0081  memory: 3937  loss: 4.4961  loss_cls: 0.6869  loss_bbox: 2.3598  loss_obj: 1.4494
03/19 19:34:05 - mmengine - INFO - Epoch(train)  [25][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:33:02  time: 0.2065  data_time: 0.0082  memory: 3639  loss: 4.4869  loss_cls: 0.6970  loss_bbox: 2.3984  loss_obj: 1.3915
03/19 19:34:16 - mmengine - INFO - Epoch(train)  [25][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:51  time: 0.2163  data_time: 0.0080  memory: 3639  loss: 4.5256  loss_cls: 0.7045  loss_bbox: 2.3752  loss_obj: 1.4460
03/19 19:34:27 - mmengine - INFO - Epoch(train)  [25][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:41  time: 0.2198  data_time: 0.0081  memory: 3639  loss: 4.5182  loss_cls: 0.7014  loss_bbox: 2.3770  loss_obj: 1.4398
03/19 19:34:39 - mmengine - INFO - Epoch(train)  [25][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:35  time: 0.2523  data_time: 0.0079  memory: 3937  loss: 4.5179  loss_cls: 0.6944  loss_bbox: 2.3630  loss_obj: 1.4605
03/19 19:34:50 - mmengine - INFO - Epoch(train)  [25][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:24  time: 0.2161  data_time: 0.0081  memory: 3639  loss: 4.5111  loss_cls: 0.6993  loss_bbox: 2.3641  loss_obj: 1.4476
03/19 19:35:01 - mmengine - INFO - Epoch(train)  [25][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:13  time: 0.2191  data_time: 0.0079  memory: 3937  loss: 4.4109  loss_cls: 0.6862  loss_bbox: 2.3202  loss_obj: 1.4045
03/19 19:35:13 - mmengine - INFO - Epoch(train)  [25][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:32:05  time: 0.2319  data_time: 0.0080  memory: 3639  loss: 4.4493  loss_cls: 0.6916  loss_bbox: 2.3381  loss_obj: 1.4197
03/19 19:35:22 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:35:22 - mmengine - INFO - Epoch(train)  [25][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:31:49  time: 0.1876  data_time: 0.0082  memory: 2587  loss: 4.4903  loss_cls: 0.7083  loss_bbox: 2.3820  loss_obj: 1.4000
03/19 19:35:22 - mmengine - INFO - Saving checkpoint at 25 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:35:27 - mmengine - INFO - Epoch(val)  [25][ 50/250]    eta: 0:00:11  time: 0.0588  data_time: 0.0073  memory: 527  
03/19 19:35:30 - mmengine - INFO - Epoch(val)  [25][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 19:35:33 - mmengine - INFO - Epoch(val)  [25][150/250]    eta: 0:00:05  time: 0.0594  data_time: 0.0066  memory: 527  
03/19 19:35:36 - mmengine - INFO - Epoch(val)  [25][200/250]    eta: 0:00:02  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 19:35:39 - mmengine - INFO - Epoch(val)  [25][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0065  memory: 527  
03/19 19:35:41 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.29s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.50s).
Accumulating evaluation results...
DONE (t=2.37s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.202
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.488
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.265
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.388
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.242
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.377
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.459
03/19 19:35:52 - mmengine - INFO - bbox_mAP_copypaste: 0.202 0.488 0.132 0.122 0.265 0.388
03/19 19:35:52 - mmengine - INFO - Epoch(val) [25][250/250]    coco/bbox_mAP: 0.2020  coco/bbox_mAP_50: 0.4880  coco/bbox_mAP_75: 0.1320  coco/bbox_mAP_s: 0.1220  coco/bbox_mAP_m: 0.2650  coco/bbox_mAP_l: 0.3880  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:36:03 - mmengine - INFO - Epoch(train)  [26][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:31:37  time: 0.2058  data_time: 0.0186  memory: 3937  loss: 4.4759  loss_cls: 0.7044  loss_bbox: 2.3748  loss_obj: 1.3967
03/19 19:36:14 - mmengine - INFO - Epoch(train)  [26][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:31:26  time: 0.2207  data_time: 0.0080  memory: 3937  loss: 4.4371  loss_cls: 0.6927  loss_bbox: 2.3499  loss_obj: 1.3944
03/19 19:36:25 - mmengine - INFO - Epoch(train)  [26][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:31:17  time: 0.2321  data_time: 0.0080  memory: 3937  loss: 4.4367  loss_cls: 0.6892  loss_bbox: 2.3368  loss_obj: 1.4107
03/19 19:36:37 - mmengine - INFO - Epoch(train)  [26][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:31:09  time: 0.2330  data_time: 0.0080  memory: 3937  loss: 4.4842  loss_cls: 0.6922  loss_bbox: 2.3757  loss_obj: 1.4162
03/19 19:36:48 - mmengine - INFO - Epoch(train)  [26][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:57  time: 0.2113  data_time: 0.0079  memory: 3639  loss: 4.5389  loss_cls: 0.7053  loss_bbox: 2.3924  loss_obj: 1.4413
03/19 19:36:59 - mmengine - INFO - Epoch(train)  [26][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:49  time: 0.2334  data_time: 0.0082  memory: 3639  loss: 4.4123  loss_cls: 0.6890  loss_bbox: 2.3204  loss_obj: 1.4029
03/19 19:37:11 - mmengine - INFO - Epoch(train)  [26][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:39  time: 0.2270  data_time: 0.0080  memory: 3639  loss: 4.4905  loss_cls: 0.6894  loss_bbox: 2.3685  loss_obj: 1.4327
03/19 19:37:22 - mmengine - INFO - Epoch(train)  [26][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:30  time: 0.2275  data_time: 0.0081  memory: 3937  loss: 4.3877  loss_cls: 0.6788  loss_bbox: 2.3265  loss_obj: 1.3825
03/19 19:37:33 - mmengine - INFO - Epoch(train)  [26][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:19  time: 0.2209  data_time: 0.0081  memory: 3937  loss: 4.5487  loss_cls: 0.7056  loss_bbox: 2.3880  loss_obj: 1.4552
03/19 19:37:44 - mmengine - INFO - Epoch(train)  [26][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:30:08  time: 0.2172  data_time: 0.0081  memory: 3639  loss: 4.4788  loss_cls: 0.6985  loss_bbox: 2.3845  loss_obj: 1.3958
03/19 19:37:55 - mmengine - INFO - Epoch(train)  [26][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:57  time: 0.2147  data_time: 0.0081  memory: 3639  loss: 4.4811  loss_cls: 0.6974  loss_bbox: 2.3731  loss_obj: 1.4106
03/19 19:38:05 - mmengine - INFO - Epoch(train)  [26][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:44  time: 0.2055  data_time: 0.0081  memory: 3071  loss: 4.4168  loss_cls: 0.6975  loss_bbox: 2.3534  loss_obj: 1.3659
03/19 19:38:15 - mmengine - INFO - Epoch(train)  [26][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:32  time: 0.2085  data_time: 0.0081  memory: 3071  loss: 4.4810  loss_cls: 0.7043  loss_bbox: 2.3753  loss_obj: 1.4014
03/19 19:38:27 - mmengine - INFO - Epoch(train)  [26][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:22  time: 0.2238  data_time: 0.0080  memory: 3937  loss: 4.4740  loss_cls: 0.6973  loss_bbox: 2.3719  loss_obj: 1.4048
03/19 19:38:39 - mmengine - INFO - Epoch(train)  [26][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:15  time: 0.2440  data_time: 0.0081  memory: 3937  loss: 4.4343  loss_cls: 0.6835  loss_bbox: 2.3573  loss_obj: 1.3935
03/19 19:38:50 - mmengine - INFO - Epoch(train)  [26][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:29:04  time: 0.2143  data_time: 0.0079  memory: 3071  loss: 4.4895  loss_cls: 0.6986  loss_bbox: 2.3643  loss_obj: 1.4265
03/19 19:39:00 - mmengine - INFO - Epoch(train)  [26][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:28:52  time: 0.2119  data_time: 0.0080  memory: 3639  loss: 4.4852  loss_cls: 0.7000  loss_bbox: 2.3723  loss_obj: 1.4129
03/19 19:39:10 - mmengine - INFO - Epoch(train)  [26][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:28:39  time: 0.2002  data_time: 0.0080  memory: 3071  loss: 4.4940  loss_cls: 0.7099  loss_bbox: 2.3775  loss_obj: 1.4066
03/19 19:39:20 - mmengine - INFO - Epoch(train)  [26][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:28:25  time: 0.2005  data_time: 0.0080  memory: 3071  loss: 4.4879  loss_cls: 0.7072  loss_bbox: 2.3930  loss_obj: 1.3877
03/19 19:39:30 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:39:30 - mmengine - INFO - Epoch(train)  [26][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:28:11  time: 0.1914  data_time: 0.0079  memory: 2587  loss: 4.4536  loss_cls: 0.7076  loss_bbox: 2.3785  loss_obj: 1.3675
03/19 19:39:30 - mmengine - INFO - Saving checkpoint at 26 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:39:35 - mmengine - INFO - Epoch(val)  [26][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/19 19:39:38 - mmengine - INFO - Epoch(val)  [26][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 19:39:41 - mmengine - INFO - Epoch(val)  [26][150/250]    eta: 0:00:05  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 19:39:44 - mmengine - INFO - Epoch(val)  [26][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 19:39:47 - mmengine - INFO - Epoch(val)  [26][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 19:39:48 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.44s).
Accumulating evaluation results...
DONE (t=2.36s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.203
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.489
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.123
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.244
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.370
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.462
03/19 19:40:00 - mmengine - INFO - bbox_mAP_copypaste: 0.203 0.489 0.135 0.123 0.264 0.392
03/19 19:40:00 - mmengine - INFO - Epoch(val) [26][250/250]    coco/bbox_mAP: 0.2030  coco/bbox_mAP_50: 0.4890  coco/bbox_mAP_75: 0.1350  coco/bbox_mAP_s: 0.1230  coco/bbox_mAP_m: 0.2640  coco/bbox_mAP_l: 0.3920  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:40:11 - mmengine - INFO - Epoch(train)  [27][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:28:00  time: 0.2154  data_time: 0.0181  memory: 3639  loss: 4.4681  loss_cls: 0.7036  loss_bbox: 2.3821  loss_obj: 1.3825
03/19 19:40:21 - mmengine - INFO - Epoch(train)  [27][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:27:47  time: 0.2050  data_time: 0.0081  memory: 2817  loss: 4.4749  loss_cls: 0.7036  loss_bbox: 2.3908  loss_obj: 1.3804
03/19 19:40:31 - mmengine - INFO - Epoch(train)  [27][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:27:32  time: 0.1904  data_time: 0.0082  memory: 2337  loss: 4.5092  loss_cls: 0.7022  loss_bbox: 2.3971  loss_obj: 1.4099
03/19 19:40:40 - mmengine - INFO - Epoch(train)  [27][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:27:19  time: 0.1977  data_time: 0.0080  memory: 2587  loss: 4.5010  loss_cls: 0.7067  loss_bbox: 2.3869  loss_obj: 1.4074
03/19 19:40:51 - mmengine - INFO - Epoch(train)  [27][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:27:07  time: 0.2121  data_time: 0.0082  memory: 3937  loss: 4.4956  loss_cls: 0.7060  loss_bbox: 2.3840  loss_obj: 1.4055
03/19 19:41:02 - mmengine - INFO - Epoch(train)  [27][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:56  time: 0.2160  data_time: 0.0081  memory: 3639  loss: 4.5095  loss_cls: 0.7107  loss_bbox: 2.3902  loss_obj: 1.4086
03/19 19:41:13 - mmengine - INFO - Epoch(train)  [27][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:47  time: 0.2297  data_time: 0.0080  memory: 3937  loss: 4.5057  loss_cls: 0.6927  loss_bbox: 2.3809  loss_obj: 1.4321
03/19 19:41:26 - mmengine - INFO - Epoch(train)  [27][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:40  time: 0.2477  data_time: 0.0080  memory: 3937  loss: 4.4991  loss_cls: 0.6940  loss_bbox: 2.3601  loss_obj: 1.4449
03/19 19:41:39 - mmengine - INFO - Epoch(train)  [27][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:36  time: 0.2614  data_time: 0.0080  memory: 3937  loss: 4.5294  loss_cls: 0.6994  loss_bbox: 2.3716  loss_obj: 1.4583
03/19 19:41:50 - mmengine - INFO - Epoch(train)  [27][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:25  time: 0.2183  data_time: 0.0080  memory: 3937  loss: 4.5155  loss_cls: 0.7040  loss_bbox: 2.3807  loss_obj: 1.4308
03/19 19:42:00 - mmengine - INFO - Epoch(train)  [27][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:13  time: 0.2118  data_time: 0.0082  memory: 3639  loss: 4.4968  loss_cls: 0.7030  loss_bbox: 2.3948  loss_obj: 1.3991
03/19 19:42:11 - mmengine - INFO - Epoch(train)  [27][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:26:02  time: 0.2180  data_time: 0.0081  memory: 3357  loss: 4.4724  loss_cls: 0.6986  loss_bbox: 2.3668  loss_obj: 1.4070
03/19 19:42:21 - mmengine - INFO - Epoch(train)  [27][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:49  time: 0.2007  data_time: 0.0081  memory: 3071  loss: 4.5010  loss_cls: 0.7118  loss_bbox: 2.3863  loss_obj: 1.4029
03/19 19:42:32 - mmengine - INFO - Epoch(train)  [27][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:37  time: 0.2103  data_time: 0.0080  memory: 3071  loss: 4.4837  loss_cls: 0.6953  loss_bbox: 2.3854  loss_obj: 1.4030
03/19 19:42:43 - mmengine - INFO - Epoch(train)  [27][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:28  time: 0.2310  data_time: 0.0081  memory: 3937  loss: 4.3931  loss_cls: 0.6887  loss_bbox: 2.3426  loss_obj: 1.3619
03/19 19:42:55 - mmengine - INFO - Epoch(train)  [27][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:19  time: 0.2263  data_time: 0.0081  memory: 3639  loss: 4.4727  loss_cls: 0.6860  loss_bbox: 2.3670  loss_obj: 1.4197
03/19 19:43:05 - mmengine - INFO - Epoch(train)  [27][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:07  time: 0.2109  data_time: 0.0082  memory: 3639  loss: 4.4315  loss_cls: 0.7010  loss_bbox: 2.3629  loss_obj: 1.3676
03/19 19:43:18 - mmengine - INFO - Epoch(train)  [27][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:25:00  time: 0.2490  data_time: 0.0081  memory: 3937  loss: 4.5060  loss_cls: 0.7068  loss_bbox: 2.3584  loss_obj: 1.4408
03/19 19:43:28 - mmengine - INFO - Epoch(train)  [27][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:24:47  time: 0.1960  data_time: 0.0082  memory: 3071  loss: 4.5319  loss_cls: 0.7167  loss_bbox: 2.3897  loss_obj: 1.4255
03/19 19:43:37 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:43:37 - mmengine - INFO - Epoch(train)  [27][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:24:32  time: 0.1934  data_time: 0.0081  memory: 3357  loss: 4.4908  loss_cls: 0.7026  loss_bbox: 2.3948  loss_obj: 1.3934
03/19 19:43:37 - mmengine - INFO - Saving checkpoint at 27 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:43:43 - mmengine - INFO - Epoch(val)  [27][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 19:43:45 - mmengine - INFO - Epoch(val)  [27][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 19:43:48 - mmengine - INFO - Epoch(val)  [27][150/250]    eta: 0:00:05  time: 0.0593  data_time: 0.0065  memory: 527  
03/19 19:43:51 - mmengine - INFO - Epoch(val)  [27][200/250]    eta: 0:00:02  time: 0.0580  data_time: 0.0064  memory: 527  
03/19 19:43:54 - mmengine - INFO - Epoch(val)  [27][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0065  memory: 527  
03/19 19:43:56 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.29s).
Accumulating evaluation results...
DONE (t=2.32s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.204
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.487
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.124
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.263
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.246
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.372
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.487
03/19 19:44:07 - mmengine - INFO - bbox_mAP_copypaste: 0.204 0.487 0.134 0.124 0.263 0.392
03/19 19:44:07 - mmengine - INFO - Epoch(val) [27][250/250]    coco/bbox_mAP: 0.2040  coco/bbox_mAP_50: 0.4870  coco/bbox_mAP_75: 0.1340  coco/bbox_mAP_s: 0.1240  coco/bbox_mAP_m: 0.2630  coco/bbox_mAP_l: 0.3920  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:44:19 - mmengine - INFO - Epoch(train)  [28][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:24:24  time: 0.2334  data_time: 0.0183  memory: 3937  loss: 4.4691  loss_cls: 0.6929  loss_bbox: 2.3647  loss_obj: 1.4116
03/19 19:44:31 - mmengine - INFO - Epoch(train)  [28][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:24:16  time: 0.2438  data_time: 0.0080  memory: 3937  loss: 4.4271  loss_cls: 0.6855  loss_bbox: 2.3248  loss_obj: 1.4168
03/19 19:44:42 - mmengine - INFO - Epoch(train)  [28][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:24:04  time: 0.2102  data_time: 0.0080  memory: 3357  loss: 4.3791  loss_cls: 0.6930  loss_bbox: 2.3198  loss_obj: 1.3663
03/19 19:44:53 - mmengine - INFO - Epoch(train)  [28][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:56  time: 0.2327  data_time: 0.0079  memory: 3639  loss: 4.4367  loss_cls: 0.6905  loss_bbox: 2.3444  loss_obj: 1.4018
03/19 19:45:05 - mmengine - INFO - Epoch(train)  [28][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:47  time: 0.2312  data_time: 0.0080  memory: 3357  loss: 4.5140  loss_cls: 0.6987  loss_bbox: 2.3787  loss_obj: 1.4366
03/19 19:45:14 - mmengine - INFO - Epoch(train)  [28][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:32  time: 0.1913  data_time: 0.0082  memory: 2337  loss: 4.4239  loss_cls: 0.6939  loss_bbox: 2.3720  loss_obj: 1.3579
03/19 19:45:27 - mmengine - INFO - Epoch(train)  [28][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:25  time: 0.2451  data_time: 0.0081  memory: 3937  loss: 4.4561  loss_cls: 0.6868  loss_bbox: 2.3612  loss_obj: 1.4081
03/19 19:45:37 - mmengine - INFO - Epoch(train)  [28][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:13  time: 0.2073  data_time: 0.0082  memory: 3937  loss: 4.4784  loss_cls: 0.6916  loss_bbox: 2.3693  loss_obj: 1.4175
03/19 19:45:48 - mmengine - INFO - Epoch(train)  [28][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:23:01  time: 0.2144  data_time: 0.0081  memory: 3357  loss: 4.4694  loss_cls: 0.6988  loss_bbox: 2.3631  loss_obj: 1.4075
03/19 19:45:59 - mmengine - INFO - Epoch(train)  [28][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:22:50  time: 0.2157  data_time: 0.0080  memory: 3639  loss: 4.4767  loss_cls: 0.7025  loss_bbox: 2.3678  loss_obj: 1.4064
03/19 19:46:10 - mmengine - INFO - Epoch(train)  [28][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:22:41  time: 0.2282  data_time: 0.0080  memory: 3937  loss: 4.4681  loss_cls: 0.6916  loss_bbox: 2.3645  loss_obj: 1.4120
03/19 19:46:20 - mmengine - INFO - Epoch(train)  [28][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:22:28  time: 0.2023  data_time: 0.0081  memory: 3357  loss: 4.4999  loss_cls: 0.7104  loss_bbox: 2.4077  loss_obj: 1.3818
03/19 19:46:31 - mmengine - INFO - Epoch(train)  [28][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:22:18  time: 0.2236  data_time: 0.0081  memory: 3937  loss: 4.5025  loss_cls: 0.7039  loss_bbox: 2.3629  loss_obj: 1.4357
03/19 19:46:44 - mmengine - INFO - Epoch(train)  [28][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:22:10  time: 0.2439  data_time: 0.0081  memory: 3639  loss: 4.4694  loss_cls: 0.6824  loss_bbox: 2.3445  loss_obj: 1.4425
03/19 19:46:54 - mmengine - INFO - Epoch(train)  [28][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:57  time: 0.2013  data_time: 0.0080  memory: 3357  loss: 4.4884  loss_cls: 0.7031  loss_bbox: 2.3862  loss_obj: 1.3991
03/19 19:47:05 - mmengine - INFO - Epoch(train)  [28][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:48  time: 0.2304  data_time: 0.0080  memory: 3639  loss: 4.5125  loss_cls: 0.7041  loss_bbox: 2.3675  loss_obj: 1.4409
03/19 19:47:16 - mmengine - INFO - Epoch(train)  [28][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:37  time: 0.2141  data_time: 0.0082  memory: 3639  loss: 4.4960  loss_cls: 0.6954  loss_bbox: 2.3809  loss_obj: 1.4196
03/19 19:47:27 - mmengine - INFO - Epoch(train)  [28][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:26  time: 0.2157  data_time: 0.0080  memory: 3357  loss: 4.4199  loss_cls: 0.6897  loss_bbox: 2.3427  loss_obj: 1.3875
03/19 19:47:38 - mmengine - INFO - Epoch(train)  [28][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:16  time: 0.2234  data_time: 0.0080  memory: 3639  loss: 4.4613  loss_cls: 0.6954  loss_bbox: 2.3590  loss_obj: 1.4069
03/19 19:47:49 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:47:49 - mmengine - INFO - Epoch(train)  [28][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:21:05  time: 0.2205  data_time: 0.0078  memory: 3357  loss: 4.5088  loss_cls: 0.6982  loss_bbox: 2.3824  loss_obj: 1.4282
03/19 19:47:49 - mmengine - INFO - Saving checkpoint at 28 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:47:54 - mmengine - INFO - Epoch(val)  [28][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0070  memory: 527  
03/19 19:47:57 - mmengine - INFO - Epoch(val)  [28][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 19:48:00 - mmengine - INFO - Epoch(val)  [28][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 19:48:03 - mmengine - INFO - Epoch(val)  [28][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 19:48:06 - mmengine - INFO - Epoch(val)  [28][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0064  memory: 527  
03/19 19:48:07 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.23s).
Accumulating evaluation results...
DONE (t=2.28s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.205
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.491
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.126
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.248
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.371
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.489
03/19 19:48:19 - mmengine - INFO - bbox_mAP_copypaste: 0.205 0.491 0.134 0.126 0.264 0.393
03/19 19:48:19 - mmengine - INFO - Epoch(val) [28][250/250]    coco/bbox_mAP: 0.2050  coco/bbox_mAP_50: 0.4910  coco/bbox_mAP_75: 0.1340  coco/bbox_mAP_s: 0.1260  coco/bbox_mAP_m: 0.2640  coco/bbox_mAP_l: 0.3930  data_time: 0.0065  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:48:29 - mmengine - INFO - Epoch(train)  [29][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:20:52  time: 0.2011  data_time: 0.0185  memory: 2337  loss: 4.5580  loss_cls: 0.7176  loss_bbox: 2.4243  loss_obj: 1.4161
03/19 19:48:40 - mmengine - INFO - Epoch(train)  [29][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:20:41  time: 0.2163  data_time: 0.0079  memory: 3937  loss: 4.5420  loss_cls: 0.7105  loss_bbox: 2.3802  loss_obj: 1.4513
03/19 19:48:50 - mmengine - INFO - Epoch(train)  [29][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:20:30  time: 0.2157  data_time: 0.0080  memory: 2817  loss: 4.4482  loss_cls: 0.6951  loss_bbox: 2.3522  loss_obj: 1.4009
03/19 19:49:02 - mmengine - INFO - Epoch(train)  [29][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:20:20  time: 0.2256  data_time: 0.0080  memory: 3937  loss: 4.5396  loss_cls: 0.7067  loss_bbox: 2.4043  loss_obj: 1.4286
03/19 19:49:12 - mmengine - INFO - Epoch(train)  [29][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:20:09  time: 0.2158  data_time: 0.0081  memory: 3071  loss: 4.4253  loss_cls: 0.6895  loss_bbox: 2.3583  loss_obj: 1.3774
03/19 19:49:22 - mmengine - INFO - Epoch(train)  [29][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:19:56  time: 0.2016  data_time: 0.0081  memory: 3071  loss: 4.4508  loss_cls: 0.6993  loss_bbox: 2.3666  loss_obj: 1.3849
03/19 19:49:33 - mmengine - INFO - Epoch(train)  [29][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:19:45  time: 0.2185  data_time: 0.0080  memory: 3071  loss: 4.4733  loss_cls: 0.6959  loss_bbox: 2.3737  loss_obj: 1.4037
03/19 19:49:44 - mmengine - INFO - Epoch(train)  [29][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:19:34  time: 0.2146  data_time: 0.0080  memory: 3071  loss: 4.4793  loss_cls: 0.6985  loss_bbox: 2.3789  loss_obj: 1.4020
03/19 19:49:54 - mmengine - INFO - Epoch(train)  [29][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:19:20  time: 0.1945  data_time: 0.0081  memory: 3071  loss: 4.4107  loss_cls: 0.6964  loss_bbox: 2.3703  loss_obj: 1.3440
03/19 19:50:05 - mmengine - INFO - Epoch(train)  [29][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:19:09  time: 0.2160  data_time: 0.0082  memory: 3639  loss: 4.4300  loss_cls: 0.6965  loss_bbox: 2.3456  loss_obj: 1.3879
03/19 19:50:16 - mmengine - INFO - Epoch(train)  [29][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:59  time: 0.2188  data_time: 0.0081  memory: 3639  loss: 4.4661  loss_cls: 0.6938  loss_bbox: 2.3620  loss_obj: 1.4103
03/19 19:50:27 - mmengine - INFO - Epoch(train)  [29][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:50  time: 0.2325  data_time: 0.0081  memory: 3937  loss: 4.4783  loss_cls: 0.6926  loss_bbox: 2.3498  loss_obj: 1.4359
03/19 19:50:38 - mmengine - INFO - Epoch(train)  [29][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:37  time: 0.2053  data_time: 0.0080  memory: 3357  loss: 4.4460  loss_cls: 0.7072  loss_bbox: 2.3480  loss_obj: 1.3908
03/19 19:50:49 - mmengine - INFO - Epoch(train)  [29][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:27  time: 0.2187  data_time: 0.0081  memory: 3937  loss: 4.6031  loss_cls: 0.7153  loss_bbox: 2.4145  loss_obj: 1.4734
03/19 19:51:00 - mmengine - INFO - Epoch(train)  [29][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:18  time: 0.2334  data_time: 0.0080  memory: 3937  loss: 4.4873  loss_cls: 0.6984  loss_bbox: 2.3530  loss_obj: 1.4360
03/19 19:51:11 - mmengine - INFO - Epoch(train)  [29][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:18:06  time: 0.2092  data_time: 0.0082  memory: 3357  loss: 4.4629  loss_cls: 0.7047  loss_bbox: 2.3738  loss_obj: 1.3845
03/19 19:51:22 - mmengine - INFO - Epoch(train)  [29][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:17:55  time: 0.2203  data_time: 0.0081  memory: 3639  loss: 4.4473  loss_cls: 0.7029  loss_bbox: 2.3456  loss_obj: 1.3988
03/19 19:51:33 - mmengine - INFO - Epoch(train)  [29][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:17:46  time: 0.2339  data_time: 0.0081  memory: 3937  loss: 4.5103  loss_cls: 0.6973  loss_bbox: 2.3728  loss_obj: 1.4401
03/19 19:51:42 - mmengine - INFO - Epoch(train)  [29][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:17:31  time: 0.1809  data_time: 0.0083  memory: 2115  loss: 4.4987  loss_cls: 0.7128  loss_bbox: 2.3975  loss_obj: 1.3884
03/19 19:51:52 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:51:52 - mmengine - INFO - Epoch(train)  [29][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:17:18  time: 0.1980  data_time: 0.0080  memory: 2817  loss: 4.5343  loss_cls: 0.7104  loss_bbox: 2.4194  loss_obj: 1.4045
03/19 19:51:52 - mmengine - INFO - Saving checkpoint at 29 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:51:58 - mmengine - INFO - Epoch(val)  [29][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0073  memory: 527  
03/19 19:52:01 - mmengine - INFO - Epoch(val)  [29][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 19:52:04 - mmengine - INFO - Epoch(val)  [29][150/250]    eta: 0:00:05  time: 0.0579  data_time: 0.0064  memory: 527  
03/19 19:52:07 - mmengine - INFO - Epoch(val)  [29][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 19:52:09 - mmengine - INFO - Epoch(val)  [29][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0065  memory: 527  
03/19 19:52:11 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.13s).
Accumulating evaluation results...
DONE (t=2.24s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.206
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.494
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.126
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.265
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.387
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.248
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.369
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.490
03/19 19:52:22 - mmengine - INFO - bbox_mAP_copypaste: 0.206 0.494 0.135 0.126 0.265 0.387
03/19 19:52:22 - mmengine - INFO - Epoch(val) [29][250/250]    coco/bbox_mAP: 0.2060  coco/bbox_mAP_50: 0.4940  coco/bbox_mAP_75: 0.1350  coco/bbox_mAP_s: 0.1260  coco/bbox_mAP_m: 0.2650  coco/bbox_mAP_l: 0.3870  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:52:32 - mmengine - INFO - Epoch(train)  [30][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:17:04  time: 0.1971  data_time: 0.0188  memory: 2337  loss: 4.5428  loss_cls: 0.7001  loss_bbox: 2.4242  loss_obj: 1.4184
03/19 19:52:44 - mmengine - INFO - Epoch(train)  [30][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:16:57  time: 0.2422  data_time: 0.0080  memory: 3937  loss: 4.4441  loss_cls: 0.6957  loss_bbox: 2.3595  loss_obj: 1.3889
03/19 19:52:55 - mmengine - INFO - Epoch(train)  [30][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:16:46  time: 0.2205  data_time: 0.0081  memory: 3937  loss: 4.5293  loss_cls: 0.7017  loss_bbox: 2.3746  loss_obj: 1.4530
03/19 19:53:05 - mmengine - INFO - Epoch(train)  [30][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:16:33  time: 0.2023  data_time: 0.0082  memory: 3071  loss: 4.4163  loss_cls: 0.7001  loss_bbox: 2.3610  loss_obj: 1.3552
03/19 19:53:16 - mmengine - INFO - Epoch(train)  [30][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:16:22  time: 0.2138  data_time: 0.0080  memory: 3639  loss: 4.4492  loss_cls: 0.6962  loss_bbox: 2.3615  loss_obj: 1.3915
03/19 19:53:25 - mmengine - INFO - Epoch(train)  [30][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:16:07  time: 0.1797  data_time: 0.0084  memory: 2115  loss: 4.5440  loss_cls: 0.7146  loss_bbox: 2.4358  loss_obj: 1.3935
03/19 19:53:36 - mmengine - INFO - Epoch(train)  [30][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:15:57  time: 0.2298  data_time: 0.0081  memory: 3937  loss: 4.3900  loss_cls: 0.6818  loss_bbox: 2.3356  loss_obj: 1.3727
03/19 19:53:47 - mmengine - INFO - Epoch(train)  [30][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:15:45  time: 0.2073  data_time: 0.0081  memory: 3357  loss: 4.4347  loss_cls: 0.6895  loss_bbox: 2.3606  loss_obj: 1.3846
03/19 19:53:56 - mmengine - INFO - Epoch(train)  [30][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:15:31  time: 0.1904  data_time: 0.0081  memory: 2337  loss: 4.4025  loss_cls: 0.6941  loss_bbox: 2.3572  loss_obj: 1.3512
03/19 19:54:07 - mmengine - INFO - Epoch(train)  [30][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:15:20  time: 0.2182  data_time: 0.0080  memory: 3639  loss: 4.3926  loss_cls: 0.6801  loss_bbox: 2.3403  loss_obj: 1.3721
03/19 19:54:18 - mmengine - INFO - Epoch(train)  [30][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:15:08  time: 0.2081  data_time: 0.0082  memory: 3639  loss: 4.5185  loss_cls: 0.7007  loss_bbox: 2.4095  loss_obj: 1.4083
03/19 19:54:28 - mmengine - INFO - Epoch(train)  [30][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:57  time: 0.2144  data_time: 0.0079  memory: 3639  loss: 4.4272  loss_cls: 0.6882  loss_bbox: 2.3617  loss_obj: 1.3773
03/19 19:54:39 - mmengine - INFO - Epoch(train)  [30][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:46  time: 0.2173  data_time: 0.0081  memory: 3937  loss: 4.4605  loss_cls: 0.6862  loss_bbox: 2.3801  loss_obj: 1.3943
03/19 19:54:50 - mmengine - INFO - Epoch(train)  [30][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:35  time: 0.2134  data_time: 0.0082  memory: 3639  loss: 4.4933  loss_cls: 0.6923  loss_bbox: 2.3945  loss_obj: 1.4065
03/19 19:55:01 - mmengine - INFO - Epoch(train)  [30][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:25  time: 0.2276  data_time: 0.0081  memory: 3639  loss: 4.4051  loss_cls: 0.6835  loss_bbox: 2.3337  loss_obj: 1.3879
03/19 19:55:13 - mmengine - INFO - Epoch(train)  [30][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:16  time: 0.2283  data_time: 0.0082  memory: 3639  loss: 4.4523  loss_cls: 0.6911  loss_bbox: 2.3560  loss_obj: 1.4053
03/19 19:55:24 - mmengine - INFO - Epoch(train)  [30][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:14:05  time: 0.2192  data_time: 0.0081  memory: 3937  loss: 4.4148  loss_cls: 0.6948  loss_bbox: 2.3558  loss_obj: 1.3643
03/19 19:55:35 - mmengine - INFO - Epoch(train)  [30][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:54  time: 0.2153  data_time: 0.0081  memory: 3639  loss: 4.4448  loss_cls: 0.6938  loss_bbox: 2.3610  loss_obj: 1.3900
03/19 19:55:45 - mmengine - INFO - Epoch(train)  [30][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:42  time: 0.2113  data_time: 0.0082  memory: 3357  loss: 4.5311  loss_cls: 0.7060  loss_bbox: 2.3833  loss_obj: 1.4418
03/19 19:55:55 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 19:55:55 - mmengine - INFO - Epoch(train)  [30][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:30  time: 0.2034  data_time: 0.0083  memory: 3937  loss: 4.4895  loss_cls: 0.6970  loss_bbox: 2.3931  loss_obj: 1.3994
03/19 19:55:55 - mmengine - INFO - Saving checkpoint at 30 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:56:01 - mmengine - INFO - Epoch(val)  [30][ 50/250]    eta: 0:00:12  time: 0.0600  data_time: 0.0071  memory: 527  
03/19 19:56:04 - mmengine - INFO - Epoch(val)  [30][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 19:56:07 - mmengine - INFO - Epoch(val)  [30][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0064  memory: 527  
03/19 19:56:10 - mmengine - INFO - Epoch(val)  [30][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 19:56:12 - mmengine - INFO - Epoch(val)  [30][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0064  memory: 527  
03/19 19:56:14 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=8.11s).
Accumulating evaluation results...
DONE (t=2.23s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.206
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.492
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.127
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.266
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.389
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.371
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.494
03/19 19:56:25 - mmengine - INFO - bbox_mAP_copypaste: 0.206 0.492 0.137 0.127 0.266 0.389
03/19 19:56:25 - mmengine - INFO - Epoch(val) [30][250/250]    coco/bbox_mAP: 0.2060  coco/bbox_mAP_50: 0.4920  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1270  coco/bbox_mAP_m: 0.2660  coco/bbox_mAP_l: 0.3890  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 19:56:37 - mmengine - INFO - Epoch(train)  [31][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:22  time: 0.2471  data_time: 0.0186  memory: 3357  loss: 4.4316  loss_cls: 0.6914  loss_bbox: 2.3506  loss_obj: 1.3896
03/19 19:56:49 - mmengine - INFO - Epoch(train)  [31][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:13  time: 0.2283  data_time: 0.0081  memory: 3639  loss: 4.4363  loss_cls: 0.6927  loss_bbox: 2.3435  loss_obj: 1.4002
03/19 19:57:00 - mmengine - INFO - Epoch(train)  [31][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:13:04  time: 0.2338  data_time: 0.0080  memory: 3937  loss: 4.4966  loss_cls: 0.7022  loss_bbox: 2.3703  loss_obj: 1.4242
03/19 19:57:12 - mmengine - INFO - Epoch(train)  [31][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:12:54  time: 0.2240  data_time: 0.0080  memory: 3639  loss: 4.4561  loss_cls: 0.6947  loss_bbox: 2.3506  loss_obj: 1.4108
03/19 19:57:22 - mmengine - INFO - Epoch(train)  [31][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:12:42  time: 0.2106  data_time: 0.0079  memory: 3357  loss: 4.3618  loss_cls: 0.6835  loss_bbox: 2.3404  loss_obj: 1.3378
03/19 19:57:33 - mmengine - INFO - Epoch(train)  [31][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:12:32  time: 0.2204  data_time: 0.0079  memory: 3937  loss: 4.3933  loss_cls: 0.6892  loss_bbox: 2.3442  loss_obj: 1.3600
03/19 19:57:44 - mmengine - INFO - Epoch(train)  [31][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:12:20  time: 0.2104  data_time: 0.0081  memory: 3639  loss: 4.4846  loss_cls: 0.7074  loss_bbox: 2.3801  loss_obj: 1.3971
03/19 19:57:54 - mmengine - INFO - Epoch(train)  [31][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:12:08  time: 0.2041  data_time: 0.0083  memory: 3937  loss: 4.4289  loss_cls: 0.6985  loss_bbox: 2.3603  loss_obj: 1.3700
03/19 19:58:05 - mmengine - INFO - Epoch(train)  [31][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:56  time: 0.2113  data_time: 0.0081  memory: 3937  loss: 4.5258  loss_cls: 0.7085  loss_bbox: 2.4053  loss_obj: 1.4121
03/19 19:58:15 - mmengine - INFO - Epoch(train)  [31][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:44  time: 0.2076  data_time: 0.0081  memory: 3357  loss: 4.4344  loss_cls: 0.6906  loss_bbox: 2.3726  loss_obj: 1.3713
03/19 19:58:26 - mmengine - INFO - Epoch(train)  [31][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:34  time: 0.2247  data_time: 0.0079  memory: 3937  loss: 4.5028  loss_cls: 0.7075  loss_bbox: 2.3800  loss_obj: 1.4152
03/19 19:58:37 - mmengine - INFO - Epoch(train)  [31][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:22  time: 0.2098  data_time: 0.0081  memory: 3639  loss: 4.4629  loss_cls: 0.7001  loss_bbox: 2.3642  loss_obj: 1.3986
03/19 19:58:48 - mmengine - INFO - Epoch(train)  [31][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:12  time: 0.2251  data_time: 0.0081  memory: 3937  loss: 4.4443  loss_cls: 0.6910  loss_bbox: 2.3509  loss_obj: 1.4024
03/19 19:58:58 - mmengine - INFO - Epoch(train)  [31][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:11:00  time: 0.2009  data_time: 0.0082  memory: 2817  loss: 4.4622  loss_cls: 0.6993  loss_bbox: 2.3777  loss_obj: 1.3853
03/19 19:59:08 - mmengine - INFO - Epoch(train)  [31][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:10:46  time: 0.1972  data_time: 0.0082  memory: 2817  loss: 4.4490  loss_cls: 0.6996  loss_bbox: 2.3740  loss_obj: 1.3753
03/19 19:59:19 - mmengine - INFO - Epoch(train)  [31][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:10:36  time: 0.2211  data_time: 0.0082  memory: 3937  loss: 4.4654  loss_cls: 0.6917  loss_bbox: 2.3614  loss_obj: 1.4123
03/19 19:59:31 - mmengine - INFO - Epoch(train)  [31][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:10:28  time: 0.2412  data_time: 0.0081  memory: 3937  loss: 4.4914  loss_cls: 0.6935  loss_bbox: 2.3717  loss_obj: 1.4262
03/19 19:59:41 - mmengine - INFO - Epoch(train)  [31][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:10:15  time: 0.2033  data_time: 0.0080  memory: 2817  loss: 4.4659  loss_cls: 0.7066  loss_bbox: 2.3826  loss_obj: 1.3768
03/19 19:59:52 - mmengine - INFO - Epoch(train)  [31][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:10:05  time: 0.2201  data_time: 0.0081  memory: 3937  loss: 4.4770  loss_cls: 0.7005  loss_bbox: 2.3713  loss_obj: 1.4052
03/19 20:00:03 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:00:03 - mmengine - INFO - Epoch(train)  [31][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:54  time: 0.2192  data_time: 0.0079  memory: 3071  loss: 4.3776  loss_cls: 0.6876  loss_bbox: 2.3115  loss_obj: 1.3786
03/19 20:00:03 - mmengine - INFO - Saving checkpoint at 31 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:00:09 - mmengine - INFO - Epoch(val)  [31][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/19 20:00:11 - mmengine - INFO - Epoch(val)  [31][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0066  memory: 527  
03/19 20:00:14 - mmengine - INFO - Epoch(val)  [31][150/250]    eta: 0:00:05  time: 0.0576  data_time: 0.0064  memory: 527  
03/19 20:00:17 - mmengine - INFO - Epoch(val)  [31][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:00:20 - mmengine - INFO - Epoch(val)  [31][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0065  memory: 527  
03/19 20:00:22 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.89s).
Accumulating evaluation results...
DONE (t=2.20s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.206
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.493
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.127
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.265
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.246
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.362
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.487
03/19 20:00:32 - mmengine - INFO - bbox_mAP_copypaste: 0.206 0.493 0.138 0.127 0.265 0.392
03/19 20:00:33 - mmengine - INFO - Epoch(val) [31][250/250]    coco/bbox_mAP: 0.2060  coco/bbox_mAP_50: 0.4930  coco/bbox_mAP_75: 0.1380  coco/bbox_mAP_s: 0.1270  coco/bbox_mAP_m: 0.2650  coco/bbox_mAP_l: 0.3920  data_time: 0.0066  time: 0.0582
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:00:44 - mmengine - INFO - Epoch(train)  [32][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:44  time: 0.2266  data_time: 0.0184  memory: 3357  loss: 4.3879  loss_cls: 0.6789  loss_bbox: 2.3306  loss_obj: 1.3783
03/19 20:00:56 - mmengine - INFO - Epoch(train)  [32][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:36  time: 0.2428  data_time: 0.0080  memory: 3937  loss: 4.4890  loss_cls: 0.6987  loss_bbox: 2.3713  loss_obj: 1.4190
03/19 20:01:06 - mmengine - INFO - Epoch(train)  [32][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:23  time: 0.1950  data_time: 0.0081  memory: 2817  loss: 4.4450  loss_cls: 0.6957  loss_bbox: 2.3763  loss_obj: 1.3731
03/19 20:01:17 - mmengine - INFO - Epoch(train)  [32][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:13  time: 0.2241  data_time: 0.0080  memory: 3639  loss: 4.4281  loss_cls: 0.6948  loss_bbox: 2.3447  loss_obj: 1.3885
03/19 20:01:27 - mmengine - INFO - Epoch(train)  [32][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:09:00  time: 0.2016  data_time: 0.0081  memory: 3937  loss: 4.4471  loss_cls: 0.6975  loss_bbox: 2.3757  loss_obj: 1.3740
03/19 20:01:37 - mmengine - INFO - Epoch(train)  [32][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:08:48  time: 0.2023  data_time: 0.0081  memory: 3639  loss: 4.4798  loss_cls: 0.7065  loss_bbox: 2.3885  loss_obj: 1.3848
03/19 20:01:48 - mmengine - INFO - Epoch(train)  [32][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:08:37  time: 0.2190  data_time: 0.0080  memory: 3937  loss: 4.4716  loss_cls: 0.7059  loss_bbox: 2.3829  loss_obj: 1.3828
03/19 20:01:59 - mmengine - INFO - Epoch(train)  [32][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:08:25  time: 0.2095  data_time: 0.0082  memory: 3639  loss: 4.4765  loss_cls: 0.6984  loss_bbox: 2.3735  loss_obj: 1.4047
03/19 20:02:11 - mmengine - INFO - Epoch(train)  [32][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:08:18  time: 0.2476  data_time: 0.0081  memory: 3937  loss: 4.4642  loss_cls: 0.6845  loss_bbox: 2.3503  loss_obj: 1.4295
03/19 20:02:21 - mmengine - INFO - Epoch(train)  [32][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:08:05  time: 0.1984  data_time: 0.0082  memory: 3937  loss: 4.5199  loss_cls: 0.6916  loss_bbox: 2.4326  loss_obj: 1.3958
03/19 20:02:32 - mmengine - INFO - Epoch(train)  [32][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:54  time: 0.2145  data_time: 0.0081  memory: 3639  loss: 4.5227  loss_cls: 0.7025  loss_bbox: 2.3926  loss_obj: 1.4276
03/19 20:02:44 - mmengine - INFO - Epoch(train)  [32][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:46  time: 0.2431  data_time: 0.0082  memory: 3937  loss: 4.5430  loss_cls: 0.7018  loss_bbox: 2.3858  loss_obj: 1.4554
03/19 20:02:54 - mmengine - INFO - Epoch(train)  [32][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:32  time: 0.1930  data_time: 0.0080  memory: 2337  loss: 4.5107  loss_cls: 0.7046  loss_bbox: 2.4007  loss_obj: 1.4055
03/19 20:03:04 - mmengine - INFO - Epoch(train)  [32][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:21  time: 0.2134  data_time: 0.0081  memory: 3937  loss: 4.3929  loss_cls: 0.6900  loss_bbox: 2.3460  loss_obj: 1.3568
03/19 20:03:16 - mmengine - INFO - Epoch(train)  [32][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:11  time: 0.2257  data_time: 0.0081  memory: 3639  loss: 4.4478  loss_cls: 0.6949  loss_bbox: 2.3708  loss_obj: 1.3820
03/19 20:03:27 - mmengine - INFO - Epoch(train)  [32][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:07:01  time: 0.2235  data_time: 0.0081  memory: 3357  loss: 4.4422  loss_cls: 0.6947  loss_bbox: 2.3474  loss_obj: 1.4001
03/19 20:03:37 - mmengine - INFO - Epoch(train)  [32][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:06:48  time: 0.2008  data_time: 0.0080  memory: 3071  loss: 4.4133  loss_cls: 0.6937  loss_bbox: 2.3784  loss_obj: 1.3413
03/19 20:03:48 - mmengine - INFO - Epoch(train)  [32][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:06:38  time: 0.2250  data_time: 0.0081  memory: 3937  loss: 4.5065  loss_cls: 0.6930  loss_bbox: 2.3761  loss_obj: 1.4374
03/19 20:03:59 - mmengine - INFO - Epoch(train)  [32][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:06:28  time: 0.2228  data_time: 0.0080  memory: 3937  loss: 4.4905  loss_cls: 0.6933  loss_bbox: 2.4010  loss_obj: 1.3962
03/19 20:04:11 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:04:11 - mmengine - INFO - Epoch(train)  [32][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:06:18  time: 0.2291  data_time: 0.0078  memory: 3639  loss: 4.4269  loss_cls: 0.6931  loss_bbox: 2.3397  loss_obj: 1.3941
03/19 20:04:11 - mmengine - INFO - Saving checkpoint at 32 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:04:16 - mmengine - INFO - Epoch(val)  [32][ 50/250]    eta: 0:00:11  time: 0.0598  data_time: 0.0072  memory: 527  
03/19 20:04:19 - mmengine - INFO - Epoch(val)  [32][100/250]    eta: 0:00:09  time: 0.0608  data_time: 0.0089  memory: 527  
03/19 20:04:22 - mmengine - INFO - Epoch(val)  [32][150/250]    eta: 0:00:05  time: 0.0577  data_time: 0.0064  memory: 527  
03/19 20:04:25 - mmengine - INFO - Epoch(val)  [32][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:04:28 - mmengine - INFO - Epoch(val)  [32][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0064  memory: 527  
03/19 20:04:29 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.99s).
Accumulating evaluation results...
DONE (t=2.19s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.207
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.494
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.128
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.267
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.310
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.246
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.364
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.487
03/19 20:04:40 - mmengine - INFO - bbox_mAP_copypaste: 0.207 0.494 0.138 0.128 0.267 0.391
03/19 20:04:40 - mmengine - INFO - Epoch(val) [32][250/250]    coco/bbox_mAP: 0.2070  coco/bbox_mAP_50: 0.4940  coco/bbox_mAP_75: 0.1380  coco/bbox_mAP_s: 0.1280  coco/bbox_mAP_m: 0.2670  coco/bbox_mAP_l: 0.3910  data_time: 0.0071  time: 0.0588
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:04:52 - mmengine - INFO - Epoch(train)  [33][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:06:10  time: 0.2438  data_time: 0.0189  memory: 3639  loss: 4.3796  loss_cls: 0.6928  loss_bbox: 2.3289  loss_obj: 1.3578
03/19 20:05:03 - mmengine - INFO - Epoch(train)  [33][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:58  time: 0.2103  data_time: 0.0079  memory: 3071  loss: 4.3780  loss_cls: 0.6842  loss_bbox: 2.3502  loss_obj: 1.3436
03/19 20:05:14 - mmengine - INFO - Epoch(train)  [33][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:49  time: 0.2304  data_time: 0.0082  memory: 3937  loss: 4.3533  loss_cls: 0.6802  loss_bbox: 2.3109  loss_obj: 1.3622
03/19 20:05:26 - mmengine - INFO - Epoch(train)  [33][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:40  time: 0.2324  data_time: 0.0080  memory: 3937  loss: 4.4709  loss_cls: 0.6876  loss_bbox: 2.3635  loss_obj: 1.4198
03/19 20:05:37 - mmengine - INFO - Epoch(train)  [33][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:29  time: 0.2191  data_time: 0.0081  memory: 3937  loss: 4.4661  loss_cls: 0.6871  loss_bbox: 2.3617  loss_obj: 1.4172
03/19 20:05:48 - mmengine - INFO - Epoch(train)  [33][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:19  time: 0.2298  data_time: 0.0080  memory: 3937  loss: 4.4272  loss_cls: 0.6872  loss_bbox: 2.3605  loss_obj: 1.3795
03/19 20:06:00 - mmengine - INFO - Epoch(train)  [33][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:05:09  time: 0.2209  data_time: 0.0081  memory: 3639  loss: 4.4579  loss_cls: 0.6963  loss_bbox: 2.3577  loss_obj: 1.4039
03/19 20:06:10 - mmengine - INFO - Epoch(train)  [33][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:58  time: 0.2149  data_time: 0.0081  memory: 3357  loss: 4.5636  loss_cls: 0.7058  loss_bbox: 2.3919  loss_obj: 1.4658
03/19 20:06:22 - mmengine - INFO - Epoch(train)  [33][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:48  time: 0.2306  data_time: 0.0080  memory: 3639  loss: 4.4134  loss_cls: 0.6888  loss_bbox: 2.3375  loss_obj: 1.3871
03/19 20:06:33 - mmengine - INFO - Epoch(train)  [33][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:38  time: 0.2206  data_time: 0.0081  memory: 3937  loss: 4.4351  loss_cls: 0.6911  loss_bbox: 2.3710  loss_obj: 1.3730
03/19 20:06:45 - mmengine - INFO - Epoch(train)  [33][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:30  time: 0.2511  data_time: 0.0081  memory: 3937  loss: 4.4257  loss_cls: 0.6979  loss_bbox: 2.3289  loss_obj: 1.3989
03/19 20:06:55 - mmengine - INFO - Epoch(train)  [33][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:17  time: 0.1981  data_time: 0.0081  memory: 2587  loss: 4.4688  loss_cls: 0.6955  loss_bbox: 2.3879  loss_obj: 1.3853
03/19 20:07:07 - mmengine - INFO - Epoch(train)  [33][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:04:08  time: 0.2343  data_time: 0.0081  memory: 3639  loss: 4.4372  loss_cls: 0.6890  loss_bbox: 2.3343  loss_obj: 1.4139
03/19 20:07:17 - mmengine - INFO - Epoch(train)  [33][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:03:55  time: 0.1985  data_time: 0.0082  memory: 3639  loss: 4.3942  loss_cls: 0.6812  loss_bbox: 2.3630  loss_obj: 1.3501
03/19 20:07:28 - mmengine - INFO - Epoch(train)  [33][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:03:44  time: 0.2124  data_time: 0.0082  memory: 3937  loss: 4.4399  loss_cls: 0.6934  loss_bbox: 2.3813  loss_obj: 1.3653
03/19 20:07:37 - mmengine - INFO - Epoch(train)  [33][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:03:29  time: 0.1799  data_time: 0.0083  memory: 1907  loss: 4.4314  loss_cls: 0.6932  loss_bbox: 2.3635  loss_obj: 1.3747
03/19 20:07:48 - mmengine - INFO - Epoch(train)  [33][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:03:20  time: 0.2291  data_time: 0.0080  memory: 3639  loss: 4.4335  loss_cls: 0.6893  loss_bbox: 2.3423  loss_obj: 1.4019
03/19 20:07:58 - mmengine - INFO - Epoch(train)  [33][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:03:07  time: 0.1989  data_time: 0.0081  memory: 2587  loss: 4.4783  loss_cls: 0.6993  loss_bbox: 2.3813  loss_obj: 1.3977
03/19 20:08:08 - mmengine - INFO - Epoch(train)  [33][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:54  time: 0.1982  data_time: 0.0081  memory: 2817  loss: 4.5001  loss_cls: 0.7060  loss_bbox: 2.3804  loss_obj: 1.4137
03/19 20:08:19 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:08:19 - mmengine - INFO - Epoch(train)  [33][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:43  time: 0.2152  data_time: 0.0081  memory: 3937  loss: 4.4148  loss_cls: 0.6919  loss_bbox: 2.3343  loss_obj: 1.3886
03/19 20:08:19 - mmengine - INFO - Saving checkpoint at 33 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:08:24 - mmengine - INFO - Epoch(val)  [33][ 50/250]    eta: 0:00:11  time: 0.0590  data_time: 0.0072  memory: 527  
03/19 20:08:27 - mmengine - INFO - Epoch(val)  [33][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:08:30 - mmengine - INFO - Epoch(val)  [33][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 20:08:33 - mmengine - INFO - Epoch(val)  [33][200/250]    eta: 0:00:02  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 20:08:36 - mmengine - INFO - Epoch(val)  [33][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0065  memory: 527  
03/19 20:08:37 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.96s).
Accumulating evaluation results...
DONE (t=2.18s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.208
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.497
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.139
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.129
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.268
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.365
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.494
03/19 20:08:48 - mmengine - INFO - bbox_mAP_copypaste: 0.208 0.497 0.139 0.129 0.268 0.397
03/19 20:08:48 - mmengine - INFO - Epoch(val) [33][250/250]    coco/bbox_mAP: 0.2080  coco/bbox_mAP_50: 0.4970  coco/bbox_mAP_75: 0.1390  coco/bbox_mAP_s: 0.1290  coco/bbox_mAP_m: 0.2680  coco/bbox_mAP_l: 0.3970  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:08:59 - mmengine - INFO - Epoch(train)  [34][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:33  time: 0.2251  data_time: 0.0176  memory: 3357  loss: 4.4256  loss_cls: 0.6886  loss_bbox: 2.3583  loss_obj: 1.3786
03/19 20:09:10 - mmengine - INFO - Epoch(train)  [34][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:23  time: 0.2239  data_time: 0.0080  memory: 3937  loss: 4.4405  loss_cls: 0.6941  loss_bbox: 2.3483  loss_obj: 1.3981
03/19 20:09:22 - mmengine - INFO - Epoch(train)  [34][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:13  time: 0.2284  data_time: 0.0080  memory: 3937  loss: 4.3942  loss_cls: 0.6855  loss_bbox: 2.3238  loss_obj: 1.3848
03/19 20:09:33 - mmengine - INFO - Epoch(train)  [34][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:02:02  time: 0.2174  data_time: 0.0081  memory: 3357  loss: 4.4574  loss_cls: 0.7050  loss_bbox: 2.3790  loss_obj: 1.3734
03/19 20:09:44 - mmengine - INFO - Epoch(train)  [34][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:01:53  time: 0.2314  data_time: 0.0080  memory: 3639  loss: 4.4152  loss_cls: 0.6897  loss_bbox: 2.3405  loss_obj: 1.3850
03/19 20:09:55 - mmengine - INFO - Epoch(train)  [34][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:01:42  time: 0.2159  data_time: 0.0081  memory: 3639  loss: 4.4881  loss_cls: 0.6941  loss_bbox: 2.4017  loss_obj: 1.3923
03/19 20:10:05 - mmengine - INFO - Epoch(train)  [34][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:01:29  time: 0.1976  data_time: 0.0084  memory: 3071  loss: 4.4586  loss_cls: 0.6963  loss_bbox: 2.3760  loss_obj: 1.3863
03/19 20:10:15 - mmengine - INFO - Epoch(train)  [34][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:01:16  time: 0.2001  data_time: 0.0081  memory: 3639  loss: 4.4855  loss_cls: 0.7070  loss_bbox: 2.3951  loss_obj: 1.3834
03/19 20:10:26 - mmengine - INFO - Epoch(train)  [34][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:01:06  time: 0.2210  data_time: 0.0082  memory: 3937  loss: 4.4659  loss_cls: 0.6940  loss_bbox: 2.3595  loss_obj: 1.4124
03/19 20:10:36 - mmengine - INFO - Epoch(train)  [34][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:53  time: 0.1959  data_time: 0.0082  memory: 3639  loss: 4.4596  loss_cls: 0.6952  loss_bbox: 2.3841  loss_obj: 1.3803
03/19 20:10:48 - mmengine - INFO - Epoch(train)  [34][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:44  time: 0.2357  data_time: 0.0082  memory: 3937  loss: 4.4854  loss_cls: 0.6964  loss_bbox: 2.3751  loss_obj: 1.4140
03/19 20:10:59 - mmengine - INFO - Epoch(train)  [34][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:33  time: 0.2192  data_time: 0.0081  memory: 3639  loss: 4.4113  loss_cls: 0.6923  loss_bbox: 2.3456  loss_obj: 1.3735
03/19 20:11:09 - mmengine - INFO - Epoch(train)  [34][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:21  time: 0.2027  data_time: 0.0081  memory: 3357  loss: 4.3601  loss_cls: 0.6862  loss_bbox: 2.3561  loss_obj: 1.3179
03/19 20:11:20 - mmengine - INFO - Epoch(train)  [34][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:11  time: 0.2282  data_time: 0.0080  memory: 3937  loss: 4.4891  loss_cls: 0.6919  loss_bbox: 2.3796  loss_obj: 1.4176
03/19 20:11:31 - mmengine - INFO - Epoch(train)  [34][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 4:00:00  time: 0.2204  data_time: 0.0081  memory: 3639  loss: 4.3764  loss_cls: 0.6922  loss_bbox: 2.3228  loss_obj: 1.3614
03/19 20:11:42 - mmengine - INFO - Epoch(train)  [34][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:59:49  time: 0.2135  data_time: 0.0081  memory: 3639  loss: 4.4531  loss_cls: 0.6960  loss_bbox: 2.3757  loss_obj: 1.3814
03/19 20:11:52 - mmengine - INFO - Epoch(train)  [34][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:59:37  time: 0.2069  data_time: 0.0081  memory: 2817  loss: 4.4539  loss_cls: 0.6905  loss_bbox: 2.3645  loss_obj: 1.3989
03/19 20:12:02 - mmengine - INFO - Epoch(train)  [34][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:59:24  time: 0.2002  data_time: 0.0081  memory: 2587  loss: 4.4040  loss_cls: 0.6892  loss_bbox: 2.3856  loss_obj: 1.3293
03/19 20:12:14 - mmengine - INFO - Epoch(train)  [34][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:59:14  time: 0.2246  data_time: 0.0081  memory: 3937  loss: 4.4611  loss_cls: 0.6973  loss_bbox: 2.3589  loss_obj: 1.4049
03/19 20:12:25 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:12:25 - mmengine - INFO - Epoch(train)  [34][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:59:04  time: 0.2265  data_time: 0.0081  memory: 3639  loss: 4.4395  loss_cls: 0.6899  loss_bbox: 2.3668  loss_obj: 1.3828
03/19 20:12:25 - mmengine - INFO - Saving checkpoint at 34 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:12:30 - mmengine - INFO - Epoch(val)  [34][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/19 20:12:33 - mmengine - INFO - Epoch(val)  [34][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 20:12:36 - mmengine - INFO - Epoch(val)  [34][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 20:12:39 - mmengine - INFO - Epoch(val)  [34][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 20:12:42 - mmengine - INFO - Epoch(val)  [34][250/250]    eta: 0:00:00  time: 0.0567  data_time: 0.0064  memory: 527  
03/19 20:12:43 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.73s).
Accumulating evaluation results...
DONE (t=2.18s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.208
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.497
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.129
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.254
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.389
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.311
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.366
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.493
03/19 20:12:54 - mmengine - INFO - bbox_mAP_copypaste: 0.208 0.497 0.137 0.129 0.254 0.389
03/19 20:12:54 - mmengine - INFO - Epoch(val) [34][250/250]    coco/bbox_mAP: 0.2080  coco/bbox_mAP_50: 0.4970  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1290  coco/bbox_mAP_m: 0.2540  coco/bbox_mAP_l: 0.3890  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:13:06 - mmengine - INFO - Epoch(train)  [35][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:58:55  time: 0.2293  data_time: 0.0182  memory: 3639  loss: 4.4350  loss_cls: 0.6920  loss_bbox: 2.3633  loss_obj: 1.3797
03/19 20:13:16 - mmengine - INFO - Epoch(train)  [35][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:58:43  time: 0.2135  data_time: 0.0081  memory: 3937  loss: 4.4428  loss_cls: 0.6892  loss_bbox: 2.3642  loss_obj: 1.3894
03/19 20:13:27 - mmengine - INFO - Epoch(train)  [35][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:58:33  time: 0.2179  data_time: 0.0082  memory: 3071  loss: 4.4341  loss_cls: 0.6878  loss_bbox: 2.3756  loss_obj: 1.3708
03/19 20:13:38 - mmengine - INFO - Epoch(train)  [35][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:58:22  time: 0.2150  data_time: 0.0080  memory: 3639  loss: 4.4079  loss_cls: 0.6858  loss_bbox: 2.3538  loss_obj: 1.3683
03/19 20:13:47 - mmengine - INFO - Epoch(train)  [35][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:58:08  time: 0.1902  data_time: 0.0082  memory: 2115  loss: 4.4636  loss_cls: 0.7066  loss_bbox: 2.3948  loss_obj: 1.3622
03/19 20:13:58 - mmengine - INFO - Epoch(train)  [35][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:57  time: 0.2156  data_time: 0.0080  memory: 3639  loss: 4.4364  loss_cls: 0.6919  loss_bbox: 2.3679  loss_obj: 1.3766
03/19 20:14:09 - mmengine - INFO - Epoch(train)  [35][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:47  time: 0.2219  data_time: 0.0082  memory: 3937  loss: 4.3970  loss_cls: 0.6925  loss_bbox: 2.3366  loss_obj: 1.3680
03/19 20:14:21 - mmengine - INFO - Epoch(train)  [35][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:38  time: 0.2391  data_time: 0.0081  memory: 3937  loss: 4.4333  loss_cls: 0.6810  loss_bbox: 2.3287  loss_obj: 1.4236
03/19 20:14:32 - mmengine - INFO - Epoch(train)  [35][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:27  time: 0.2188  data_time: 0.0083  memory: 3639  loss: 4.4614  loss_cls: 0.6947  loss_bbox: 2.3561  loss_obj: 1.4105
03/19 20:14:42 - mmengine - INFO - Epoch(train)  [35][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:14  time: 0.1902  data_time: 0.0080  memory: 2115  loss: 4.4620  loss_cls: 0.6987  loss_bbox: 2.3781  loss_obj: 1.3852
03/19 20:14:53 - mmengine - INFO - Epoch(train)  [35][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:57:03  time: 0.2227  data_time: 0.0080  memory: 3937  loss: 4.4822  loss_cls: 0.6972  loss_bbox: 2.3631  loss_obj: 1.4219
03/19 20:15:04 - mmengine - INFO - Epoch(train)  [35][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:56:53  time: 0.2248  data_time: 0.0080  memory: 3639  loss: 4.3616  loss_cls: 0.6786  loss_bbox: 2.3315  loss_obj: 1.3516
03/19 20:15:14 - mmengine - INFO - Epoch(train)  [35][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:56:40  time: 0.1955  data_time: 0.0082  memory: 3357  loss: 4.4795  loss_cls: 0.6984  loss_bbox: 2.3992  loss_obj: 1.3820
03/19 20:15:25 - mmengine - INFO - Epoch(train)  [35][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:56:29  time: 0.2119  data_time: 0.0081  memory: 3639  loss: 4.4202  loss_cls: 0.6947  loss_bbox: 2.3688  loss_obj: 1.3567
03/19 20:15:35 - mmengine - INFO - Epoch(train)  [35][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:56:17  time: 0.2099  data_time: 0.0081  memory: 3071  loss: 4.5189  loss_cls: 0.6970  loss_bbox: 2.4004  loss_obj: 1.4214
03/19 20:15:47 - mmengine - INFO - Epoch(train)  [35][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:56:08  time: 0.2372  data_time: 0.0080  memory: 3937  loss: 4.4548  loss_cls: 0.6894  loss_bbox: 2.3707  loss_obj: 1.3947
03/19 20:15:58 - mmengine - INFO - Epoch(train)  [35][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:58  time: 0.2290  data_time: 0.0081  memory: 3639  loss: 4.4222  loss_cls: 0.6851  loss_bbox: 2.3351  loss_obj: 1.4020
03/19 20:16:10 - mmengine - INFO - Epoch(train)  [35][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:50  time: 0.2414  data_time: 0.0079  memory: 3937  loss: 4.4101  loss_cls: 0.6842  loss_bbox: 2.3512  loss_obj: 1.3747
03/19 20:16:21 - mmengine - INFO - Epoch(train)  [35][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:38  time: 0.2046  data_time: 0.0080  memory: 3937  loss: 4.4211  loss_cls: 0.6968  loss_bbox: 2.3511  loss_obj: 1.3732
03/19 20:16:30 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:16:30 - mmengine - INFO - Epoch(train)  [35][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:24  time: 0.1842  data_time: 0.0081  memory: 2115  loss: 4.3932  loss_cls: 0.6903  loss_bbox: 2.3595  loss_obj: 1.3435
03/19 20:16:30 - mmengine - INFO - Saving checkpoint at 35 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:16:35 - mmengine - INFO - Epoch(val)  [35][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0072  memory: 527  
03/19 20:16:38 - mmengine - INFO - Epoch(val)  [35][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 20:16:41 - mmengine - INFO - Epoch(val)  [35][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:16:44 - mmengine - INFO - Epoch(val)  [35][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0066  memory: 527  
03/19 20:16:47 - mmengine - INFO - Epoch(val)  [35][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0065  memory: 527  
03/19 20:16:48 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.94s).
Accumulating evaluation results...
DONE (t=2.15s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.209
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.499
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.130
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.256
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.368
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.495
03/19 20:16:59 - mmengine - INFO - bbox_mAP_copypaste: 0.209 0.499 0.137 0.130 0.256 0.392
03/19 20:16:59 - mmengine - INFO - Epoch(val) [35][250/250]    coco/bbox_mAP: 0.2090  coco/bbox_mAP_50: 0.4990  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1300  coco/bbox_mAP_m: 0.2560  coco/bbox_mAP_l: 0.3920  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:17:11 - mmengine - INFO - Epoch(train)  [36][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:14  time: 0.2281  data_time: 0.0184  memory: 3937  loss: 4.4340  loss_cls: 0.6981  loss_bbox: 2.3648  loss_obj: 1.3711
03/19 20:17:21 - mmengine - INFO - Epoch(train)  [36][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:55:03  time: 0.2130  data_time: 0.0082  memory: 3357  loss: 4.4249  loss_cls: 0.6872  loss_bbox: 2.3640  loss_obj: 1.3737
03/19 20:17:32 - mmengine - INFO - Epoch(train)  [36][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:54:51  time: 0.2080  data_time: 0.0081  memory: 3357  loss: 4.3885  loss_cls: 0.6911  loss_bbox: 2.3458  loss_obj: 1.3516
03/19 20:17:42 - mmengine - INFO - Epoch(train)  [36][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:54:39  time: 0.2069  data_time: 0.0082  memory: 3357  loss: 4.4691  loss_cls: 0.6950  loss_bbox: 2.3834  loss_obj: 1.3908
03/19 20:17:54 - mmengine - INFO - Epoch(train)  [36][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:54:30  time: 0.2313  data_time: 0.0081  memory: 3639  loss: 4.4219  loss_cls: 0.6850  loss_bbox: 2.3295  loss_obj: 1.4074
03/19 20:18:04 - mmengine - INFO - Epoch(train)  [36][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:54:18  time: 0.2127  data_time: 0.0082  memory: 3639  loss: 4.3976  loss_cls: 0.6934  loss_bbox: 2.3638  loss_obj: 1.3404
03/19 20:18:15 - mmengine - INFO - Epoch(train)  [36][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:54:07  time: 0.2155  data_time: 0.0081  memory: 3937  loss: 4.3779  loss_cls: 0.6827  loss_bbox: 2.3397  loss_obj: 1.3555
03/19 20:18:27 - mmengine - INFO - Epoch(train)  [36][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:58  time: 0.2313  data_time: 0.0081  memory: 3937  loss: 4.4567  loss_cls: 0.6850  loss_bbox: 2.3793  loss_obj: 1.3924
03/19 20:18:38 - mmengine - INFO - Epoch(train)  [36][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:47  time: 0.2187  data_time: 0.0083  memory: 3937  loss: 4.4229  loss_cls: 0.6928  loss_bbox: 2.3671  loss_obj: 1.3631
03/19 20:18:47 - mmengine - INFO - Epoch(train)  [36][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:34  time: 0.1945  data_time: 0.0081  memory: 3071  loss: 4.3385  loss_cls: 0.6854  loss_bbox: 2.3408  loss_obj: 1.3123
03/19 20:18:57 - mmengine - INFO - Epoch(train)  [36][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:21  time: 0.1882  data_time: 0.0081  memory: 2337  loss: 4.3896  loss_cls: 0.6929  loss_bbox: 2.3567  loss_obj: 1.3400
03/19 20:19:10 - mmengine - INFO - Epoch(train)  [36][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:13  time: 0.2574  data_time: 0.0081  memory: 3937  loss: 4.4622  loss_cls: 0.6851  loss_bbox: 2.3430  loss_obj: 1.4341
03/19 20:19:21 - mmengine - INFO - Epoch(train)  [36][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:53:03  time: 0.2197  data_time: 0.0081  memory: 3937  loss: 4.4656  loss_cls: 0.6948  loss_bbox: 2.3639  loss_obj: 1.4070
03/19 20:19:32 - mmengine - INFO - Epoch(train)  [36][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:52:52  time: 0.2161  data_time: 0.0080  memory: 3639  loss: 4.3956  loss_cls: 0.6835  loss_bbox: 2.3470  loss_obj: 1.3651
03/19 20:19:43 - mmengine - INFO - Epoch(train)  [36][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:52:42  time: 0.2334  data_time: 0.0081  memory: 3937  loss: 4.4228  loss_cls: 0.6856  loss_bbox: 2.3577  loss_obj: 1.3795
03/19 20:19:55 - mmengine - INFO - Epoch(train)  [36][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:52:32  time: 0.2273  data_time: 0.0080  memory: 3937  loss: 4.4747  loss_cls: 0.6927  loss_bbox: 2.3628  loss_obj: 1.4192
03/19 20:20:05 - mmengine - INFO - Epoch(train)  [36][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:52:21  time: 0.2092  data_time: 0.0082  memory: 3357  loss: 4.4325  loss_cls: 0.6899  loss_bbox: 2.3546  loss_obj: 1.3880
03/19 20:20:17 - mmengine - INFO - Epoch(train)  [36][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:52:11  time: 0.2311  data_time: 0.0080  memory: 3937  loss: 4.3457  loss_cls: 0.6833  loss_bbox: 2.3231  loss_obj: 1.3393
03/19 20:20:27 - mmengine - INFO - Epoch(train)  [36][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:58  time: 0.1975  data_time: 0.0083  memory: 2817  loss: 4.2937  loss_cls: 0.6815  loss_bbox: 2.3130  loss_obj: 1.2992
03/19 20:20:36 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:20:36 - mmengine - INFO - Epoch(train)  [36][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:46  time: 0.1951  data_time: 0.0081  memory: 3357  loss: 4.4825  loss_cls: 0.6979  loss_bbox: 2.3956  loss_obj: 1.3890
03/19 20:20:36 - mmengine - INFO - Saving checkpoint at 36 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:20:42 - mmengine - INFO - Epoch(val)  [36][ 50/250]    eta: 0:00:11  time: 0.0591  data_time: 0.0072  memory: 527  
03/19 20:20:45 - mmengine - INFO - Epoch(val)  [36][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 20:20:47 - mmengine - INFO - Epoch(val)  [36][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 20:20:50 - mmengine - INFO - Epoch(val)  [36][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 20:20:53 - mmengine - INFO - Epoch(val)  [36][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0065  memory: 527  
03/19 20:20:55 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.93s).
Accumulating evaluation results...
DONE (t=2.16s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.210
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.500
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.257
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.370
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.492
03/19 20:21:05 - mmengine - INFO - bbox_mAP_copypaste: 0.210 0.500 0.137 0.132 0.257 0.392
03/19 20:21:05 - mmengine - INFO - Epoch(val) [36][250/250]    coco/bbox_mAP: 0.2100  coco/bbox_mAP_50: 0.5000  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1320  coco/bbox_mAP_m: 0.2570  coco/bbox_mAP_l: 0.3920  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:21:17 - mmengine - INFO - Epoch(train)  [37][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:35  time: 0.2219  data_time: 0.0184  memory: 3357  loss: 4.3758  loss_cls: 0.6897  loss_bbox: 2.3442  loss_obj: 1.3419
03/19 20:21:28 - mmengine - INFO - Epoch(train)  [37][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:25  time: 0.2245  data_time: 0.0082  memory: 3937  loss: 4.4772  loss_cls: 0.6959  loss_bbox: 2.3767  loss_obj: 1.4045
03/19 20:21:39 - mmengine - INFO - Epoch(train)  [37][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:14  time: 0.2213  data_time: 0.0083  memory: 3357  loss: 4.4352  loss_cls: 0.6951  loss_bbox: 2.3662  loss_obj: 1.3740
03/19 20:21:50 - mmengine - INFO - Epoch(train)  [37][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:51:03  time: 0.2161  data_time: 0.0081  memory: 3937  loss: 4.3995  loss_cls: 0.6831  loss_bbox: 2.3609  loss_obj: 1.3555
03/19 20:22:00 - mmengine - INFO - Epoch(train)  [37][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:50:51  time: 0.2049  data_time: 0.0082  memory: 3357  loss: 4.4583  loss_cls: 0.7007  loss_bbox: 2.3888  loss_obj: 1.3688
03/19 20:22:11 - mmengine - INFO - Epoch(train)  [37][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:50:40  time: 0.2125  data_time: 0.0082  memory: 3937  loss: 4.3936  loss_cls: 0.6937  loss_bbox: 2.3406  loss_obj: 1.3594
03/19 20:22:22 - mmengine - INFO - Epoch(train)  [37][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:50:29  time: 0.2189  data_time: 0.0081  memory: 3937  loss: 4.3860  loss_cls: 0.6821  loss_bbox: 2.3509  loss_obj: 1.3530
03/19 20:22:31 - mmengine - INFO - Epoch(train)  [37][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:50:17  time: 0.1960  data_time: 0.0083  memory: 3071  loss: 4.4667  loss_cls: 0.7016  loss_bbox: 2.3729  loss_obj: 1.3922
03/19 20:22:43 - mmengine - INFO - Epoch(train)  [37][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:50:06  time: 0.2227  data_time: 0.0081  memory: 3639  loss: 4.4288  loss_cls: 0.6865  loss_bbox: 2.3558  loss_obj: 1.3866
03/19 20:22:53 - mmengine - INFO - Epoch(train)  [37][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:49:54  time: 0.2027  data_time: 0.0082  memory: 2817  loss: 4.4898  loss_cls: 0.6958  loss_bbox: 2.4027  loss_obj: 1.3913
03/19 20:23:03 - mmengine - INFO - Epoch(train)  [37][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:49:43  time: 0.2092  data_time: 0.0082  memory: 3937  loss: 4.4567  loss_cls: 0.6942  loss_bbox: 2.3645  loss_obj: 1.3981
03/19 20:23:13 - mmengine - INFO - Epoch(train)  [37][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:49:30  time: 0.1938  data_time: 0.0085  memory: 3071  loss: 4.3999  loss_cls: 0.6916  loss_bbox: 2.3762  loss_obj: 1.3322
03/19 20:23:24 - mmengine - INFO - Epoch(train)  [37][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:49:19  time: 0.2162  data_time: 0.0081  memory: 3357  loss: 4.4254  loss_cls: 0.6835  loss_bbox: 2.3676  loss_obj: 1.3743
03/19 20:23:35 - mmengine - INFO - Epoch(train)  [37][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:49:09  time: 0.2288  data_time: 0.0081  memory: 3937  loss: 4.4190  loss_cls: 0.6850  loss_bbox: 2.3457  loss_obj: 1.3884
03/19 20:23:46 - mmengine - INFO - Epoch(train)  [37][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:58  time: 0.2112  data_time: 0.0081  memory: 3071  loss: 4.4479  loss_cls: 0.6964  loss_bbox: 2.3535  loss_obj: 1.3981
03/19 20:23:56 - mmengine - INFO - Epoch(train)  [37][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:45  time: 0.2016  data_time: 0.0082  memory: 3071  loss: 4.4075  loss_cls: 0.6967  loss_bbox: 2.3578  loss_obj: 1.3530
03/19 20:24:06 - mmengine - INFO - Epoch(train)  [37][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:34  time: 0.2088  data_time: 0.0082  memory: 3937  loss: 4.3974  loss_cls: 0.6859  loss_bbox: 2.3698  loss_obj: 1.3417
03/19 20:24:18 - mmengine - INFO - Epoch(train)  [37][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:24  time: 0.2318  data_time: 0.0082  memory: 3937  loss: 4.4296  loss_cls: 0.6863  loss_bbox: 2.3667  loss_obj: 1.3766
03/19 20:24:29 - mmengine - INFO - Epoch(train)  [37][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:13  time: 0.2161  data_time: 0.0082  memory: 3357  loss: 4.4333  loss_cls: 0.6797  loss_bbox: 2.3634  loss_obj: 1.3902
03/19 20:24:39 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:24:39 - mmengine - INFO - Epoch(train)  [37][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:48:02  time: 0.2118  data_time: 0.0079  memory: 2817  loss: 4.4412  loss_cls: 0.6828  loss_bbox: 2.3610  loss_obj: 1.3975
03/19 20:24:39 - mmengine - INFO - Saving checkpoint at 37 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:24:45 - mmengine - INFO - Epoch(val)  [37][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0071  memory: 527  
03/19 20:24:48 - mmengine - INFO - Epoch(val)  [37][100/250]    eta: 0:00:08  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 20:24:50 - mmengine - INFO - Epoch(val)  [37][150/250]    eta: 0:00:05  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 20:24:53 - mmengine - INFO - Epoch(val)  [37][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:24:56 - mmengine - INFO - Epoch(val)  [37][250/250]    eta: 0:00:00  time: 0.0578  data_time: 0.0065  memory: 527  
03/19 20:24:58 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.77s).
Accumulating evaluation results...
DONE (t=2.11s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.211
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.502
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.257
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.399
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.251
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.493
03/19 20:25:08 - mmengine - INFO - bbox_mAP_copypaste: 0.211 0.502 0.138 0.134 0.257 0.399
03/19 20:25:08 - mmengine - INFO - Epoch(val) [37][250/250]    coco/bbox_mAP: 0.2110  coco/bbox_mAP_50: 0.5020  coco/bbox_mAP_75: 0.1380  coco/bbox_mAP_s: 0.1340  coco/bbox_mAP_m: 0.2570  coco/bbox_mAP_l: 0.3990  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:25:20 - mmengine - INFO - Epoch(train)  [38][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:47:53  time: 0.2434  data_time: 0.0186  memory: 3937  loss: 4.4637  loss_cls: 0.6945  loss_bbox: 2.3572  loss_obj: 1.4120
03/19 20:25:31 - mmengine - INFO - Epoch(train)  [38][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:47:42  time: 0.2170  data_time: 0.0082  memory: 3639  loss: 4.4544  loss_cls: 0.6907  loss_bbox: 2.3614  loss_obj: 1.4023
03/19 20:25:42 - mmengine - INFO - Epoch(train)  [38][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:47:31  time: 0.2127  data_time: 0.0080  memory: 3357  loss: 4.4533  loss_cls: 0.6938  loss_bbox: 2.3709  loss_obj: 1.3887
03/19 20:25:53 - mmengine - INFO - Epoch(train)  [38][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:47:21  time: 0.2264  data_time: 0.0081  memory: 3937  loss: 4.4190  loss_cls: 0.6937  loss_bbox: 2.3575  loss_obj: 1.3678
03/19 20:26:04 - mmengine - INFO - Epoch(train)  [38][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:47:10  time: 0.2183  data_time: 0.0081  memory: 3639  loss: 4.3950  loss_cls: 0.6794  loss_bbox: 2.3521  loss_obj: 1.3636
03/19 20:26:15 - mmengine - INFO - Epoch(train)  [38][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:59  time: 0.2172  data_time: 0.0081  memory: 3357  loss: 4.4822  loss_cls: 0.7034  loss_bbox: 2.3871  loss_obj: 1.3916
03/19 20:26:25 - mmengine - INFO - Epoch(train)  [38][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:47  time: 0.1956  data_time: 0.0081  memory: 3639  loss: 4.4416  loss_cls: 0.7011  loss_bbox: 2.3742  loss_obj: 1.3663
03/19 20:26:35 - mmengine - INFO - Epoch(train)  [38][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:35  time: 0.2042  data_time: 0.0082  memory: 2817  loss: 4.4869  loss_cls: 0.7017  loss_bbox: 2.3970  loss_obj: 1.3882
03/19 20:26:45 - mmengine - INFO - Epoch(train)  [38][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:23  time: 0.2007  data_time: 0.0081  memory: 3639  loss: 4.3931  loss_cls: 0.6904  loss_bbox: 2.3622  loss_obj: 1.3405
03/19 20:26:55 - mmengine - INFO - Epoch(train)  [38][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:10  time: 0.1948  data_time: 0.0082  memory: 2587  loss: 4.4417  loss_cls: 0.6951  loss_bbox: 2.3458  loss_obj: 1.4008
03/19 20:27:06 - mmengine - INFO - Epoch(train)  [38][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:46:00  time: 0.2299  data_time: 0.0082  memory: 3937  loss: 4.4167  loss_cls: 0.6872  loss_bbox: 2.3247  loss_obj: 1.4048
03/19 20:27:17 - mmengine - INFO - Epoch(train)  [38][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:45:49  time: 0.2117  data_time: 0.0081  memory: 3071  loss: 4.4055  loss_cls: 0.6835  loss_bbox: 2.3403  loss_obj: 1.3817
03/19 20:27:28 - mmengine - INFO - Epoch(train)  [38][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:45:39  time: 0.2275  data_time: 0.0081  memory: 3937  loss: 4.4535  loss_cls: 0.6961  loss_bbox: 2.3613  loss_obj: 1.3961
03/19 20:27:40 - mmengine - INFO - Epoch(train)  [38][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:45:28  time: 0.2220  data_time: 0.0081  memory: 3639  loss: 4.3947  loss_cls: 0.6881  loss_bbox: 2.3467  loss_obj: 1.3599
03/19 20:27:51 - mmengine - INFO - Epoch(train)  [38][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:45:18  time: 0.2203  data_time: 0.0081  memory: 3937  loss: 4.4282  loss_cls: 0.6887  loss_bbox: 2.3552  loss_obj: 1.3844
03/19 20:28:03 - mmengine - INFO - Epoch(train)  [38][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:45:09  time: 0.2407  data_time: 0.0080  memory: 3937  loss: 4.3875  loss_cls: 0.6749  loss_bbox: 2.3142  loss_obj: 1.3984
03/19 20:28:14 - mmengine - INFO - Epoch(train)  [38][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:59  time: 0.2274  data_time: 0.0081  memory: 3639  loss: 4.3910  loss_cls: 0.6805  loss_bbox: 2.3482  loss_obj: 1.3624
03/19 20:28:26 - mmengine - INFO - Epoch(train)  [38][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:49  time: 0.2305  data_time: 0.0081  memory: 3639  loss: 4.3927  loss_cls: 0.6876  loss_bbox: 2.3469  loss_obj: 1.3581
03/19 20:28:37 - mmengine - INFO - Epoch(train)  [38][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:39  time: 0.2338  data_time: 0.0080  memory: 3937  loss: 4.4165  loss_cls: 0.6911  loss_bbox: 2.3557  loss_obj: 1.3697
03/19 20:28:46 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:28:46 - mmengine - INFO - Epoch(train)  [38][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:25  time: 0.1733  data_time: 0.0082  memory: 2115  loss: 4.4834  loss_cls: 0.7080  loss_bbox: 2.4074  loss_obj: 1.3681
03/19 20:28:46 - mmengine - INFO - Saving checkpoint at 38 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:28:51 - mmengine - INFO - Epoch(val)  [38][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/19 20:28:54 - mmengine - INFO - Epoch(val)  [38][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 20:28:57 - mmengine - INFO - Epoch(val)  [38][150/250]    eta: 0:00:05  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 20:29:00 - mmengine - INFO - Epoch(val)  [38][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 20:29:03 - mmengine - INFO - Epoch(val)  [38][250/250]    eta: 0:00:00  time: 0.0569  data_time: 0.0065  memory: 527  
03/19 20:29:04 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.83s).
Accumulating evaluation results...
DONE (t=2.11s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.211
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.503
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.136
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.259
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.251
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.378
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.492
03/19 20:29:15 - mmengine - INFO - bbox_mAP_copypaste: 0.211 0.503 0.136 0.134 0.259 0.391
03/19 20:29:15 - mmengine - INFO - Epoch(val) [38][250/250]    coco/bbox_mAP: 0.2110  coco/bbox_mAP_50: 0.5030  coco/bbox_mAP_75: 0.1360  coco/bbox_mAP_s: 0.1340  coco/bbox_mAP_m: 0.2590  coco/bbox_mAP_l: 0.3910  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:29:26 - mmengine - INFO - Epoch(train)  [39][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:14  time: 0.2199  data_time: 0.0185  memory: 3071  loss: 4.4063  loss_cls: 0.6912  loss_bbox: 2.3559  loss_obj: 1.3593
03/19 20:29:37 - mmengine - INFO - Epoch(train)  [39][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:44:03  time: 0.2119  data_time: 0.0081  memory: 3639  loss: 4.3952  loss_cls: 0.6773  loss_bbox: 2.3373  loss_obj: 1.3807
03/19 20:29:46 - mmengine - INFO - Epoch(train)  [39][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:43:50  time: 0.1955  data_time: 0.0083  memory: 3071  loss: 4.4333  loss_cls: 0.7021  loss_bbox: 2.3834  loss_obj: 1.3478
03/19 20:29:56 - mmengine - INFO - Epoch(train)  [39][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:43:38  time: 0.1948  data_time: 0.0082  memory: 3357  loss: 4.4556  loss_cls: 0.6988  loss_bbox: 2.3766  loss_obj: 1.3803
03/19 20:30:07 - mmengine - INFO - Epoch(train)  [39][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:43:27  time: 0.2214  data_time: 0.0080  memory: 3937  loss: 4.3675  loss_cls: 0.6793  loss_bbox: 2.3277  loss_obj: 1.3605
03/19 20:30:18 - mmengine - INFO - Epoch(train)  [39][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:43:16  time: 0.2074  data_time: 0.0081  memory: 3357  loss: 4.3819  loss_cls: 0.6815  loss_bbox: 2.3567  loss_obj: 1.3436
03/19 20:30:28 - mmengine - INFO - Epoch(train)  [39][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:43:04  time: 0.2033  data_time: 0.0082  memory: 3639  loss: 4.4649  loss_cls: 0.7006  loss_bbox: 2.3808  loss_obj: 1.3835
03/19 20:30:39 - mmengine - INFO - Epoch(train)  [39][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:42:53  time: 0.2184  data_time: 0.0081  memory: 3937  loss: 4.4596  loss_cls: 0.6930  loss_bbox: 2.3737  loss_obj: 1.3929
03/19 20:30:50 - mmengine - INFO - Epoch(train)  [39][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:42:43  time: 0.2280  data_time: 0.0081  memory: 3639  loss: 4.3490  loss_cls: 0.6795  loss_bbox: 2.3183  loss_obj: 1.3511
03/19 20:31:00 - mmengine - INFO - Epoch(train)  [39][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:42:30  time: 0.1923  data_time: 0.0083  memory: 2817  loss: 4.4259  loss_cls: 0.7083  loss_bbox: 2.3693  loss_obj: 1.3483
03/19 20:31:11 - mmengine - INFO - Epoch(train)  [39][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:42:20  time: 0.2253  data_time: 0.0081  memory: 3937  loss: 4.4924  loss_cls: 0.6917  loss_bbox: 2.3871  loss_obj: 1.4135
03/19 20:31:21 - mmengine - INFO - Epoch(train)  [39][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:42:08  time: 0.2058  data_time: 0.0081  memory: 3357  loss: 4.4447  loss_cls: 0.7070  loss_bbox: 2.3661  loss_obj: 1.3716
03/19 20:31:32 - mmengine - INFO - Epoch(train)  [39][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:57  time: 0.2154  data_time: 0.0080  memory: 3357  loss: 4.4110  loss_cls: 0.6929  loss_bbox: 2.3388  loss_obj: 1.3792
03/19 20:31:43 - mmengine - INFO - Epoch(train)  [39][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:47  time: 0.2248  data_time: 0.0080  memory: 3937  loss: 4.4225  loss_cls: 0.6882  loss_bbox: 2.3575  loss_obj: 1.3768
03/19 20:31:55 - mmengine - INFO - Epoch(train)  [39][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:37  time: 0.2275  data_time: 0.0081  memory: 3639  loss: 4.4324  loss_cls: 0.6897  loss_bbox: 2.3398  loss_obj: 1.4030
03/19 20:32:06 - mmengine - INFO - Epoch(train)  [39][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:26  time: 0.2157  data_time: 0.0083  memory: 3937  loss: 4.4779  loss_cls: 0.6963  loss_bbox: 2.3777  loss_obj: 1.4039
03/19 20:32:16 - mmengine - INFO - Epoch(train)  [39][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:14  time: 0.2090  data_time: 0.0081  memory: 3639  loss: 4.4927  loss_cls: 0.7004  loss_bbox: 2.3879  loss_obj: 1.4044
03/19 20:32:25 - mmengine - INFO - Epoch(train)  [39][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:41:01  time: 0.1873  data_time: 0.0082  memory: 2817  loss: 4.4697  loss_cls: 0.7014  loss_bbox: 2.4120  loss_obj: 1.3564
03/19 20:32:35 - mmengine - INFO - Epoch(train)  [39][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:40:49  time: 0.2012  data_time: 0.0081  memory: 3357  loss: 4.3532  loss_cls: 0.6857  loss_bbox: 2.3353  loss_obj: 1.3322
03/19 20:32:47 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:32:47 - mmengine - INFO - Epoch(train)  [39][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:40:39  time: 0.2235  data_time: 0.0080  memory: 3937  loss: 4.4049  loss_cls: 0.6817  loss_bbox: 2.3515  loss_obj: 1.3717
03/19 20:32:47 - mmengine - INFO - Saving checkpoint at 39 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:32:52 - mmengine - INFO - Epoch(val)  [39][ 50/250]    eta: 0:00:11  time: 0.0590  data_time: 0.0072  memory: 527  
03/19 20:32:55 - mmengine - INFO - Epoch(val)  [39][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 20:32:58 - mmengine - INFO - Epoch(val)  [39][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 20:33:01 - mmengine - INFO - Epoch(val)  [39][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 20:33:04 - mmengine - INFO - Epoch(val)  [39][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0064  memory: 527  
03/19 20:33:05 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.59s).
Accumulating evaluation results...
DONE (t=2.34s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.212
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.505
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.263
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.390
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.316
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.316
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.316
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.253
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.489
03/19 20:33:16 - mmengine - INFO - bbox_mAP_copypaste: 0.212 0.505 0.137 0.135 0.263 0.390
03/19 20:33:16 - mmengine - INFO - Epoch(val) [39][250/250]    coco/bbox_mAP: 0.2120  coco/bbox_mAP_50: 0.5050  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1350  coco/bbox_mAP_m: 0.2630  coco/bbox_mAP_l: 0.3900  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:33:28 - mmengine - INFO - Epoch(train)  [40][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:40:29  time: 0.2342  data_time: 0.0185  memory: 3639  loss: 4.4201  loss_cls: 0.6885  loss_bbox: 2.3728  loss_obj: 1.3587
03/19 20:33:38 - mmengine - INFO - Epoch(train)  [40][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:40:18  time: 0.2167  data_time: 0.0081  memory: 3937  loss: 4.3528  loss_cls: 0.6867  loss_bbox: 2.3108  loss_obj: 1.3553
03/19 20:33:49 - mmengine - INFO - Epoch(train)  [40][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:40:07  time: 0.2127  data_time: 0.0082  memory: 2817  loss: 4.3897  loss_cls: 0.6913  loss_bbox: 2.3536  loss_obj: 1.3449
03/19 20:34:00 - mmengine - INFO - Epoch(train)  [40][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:57  time: 0.2248  data_time: 0.0080  memory: 3937  loss: 4.4269  loss_cls: 0.6879  loss_bbox: 2.3468  loss_obj: 1.3922
03/19 20:34:12 - mmengine - INFO - Epoch(train)  [40][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:47  time: 0.2341  data_time: 0.0080  memory: 3937  loss: 4.4133  loss_cls: 0.6756  loss_bbox: 2.3479  loss_obj: 1.3897
03/19 20:34:23 - mmengine - INFO - Epoch(train)  [40][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:37  time: 0.2200  data_time: 0.0081  memory: 3639  loss: 4.3794  loss_cls: 0.6838  loss_bbox: 2.3333  loss_obj: 1.3623
03/19 20:34:33 - mmengine - INFO - Epoch(train)  [40][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:25  time: 0.2036  data_time: 0.0081  memory: 3937  loss: 4.3889  loss_cls: 0.6888  loss_bbox: 2.3686  loss_obj: 1.3315
03/19 20:34:44 - mmengine - INFO - Epoch(train)  [40][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:14  time: 0.2163  data_time: 0.0080  memory: 3639  loss: 4.4310  loss_cls: 0.6829  loss_bbox: 2.3725  loss_obj: 1.3756
03/19 20:34:55 - mmengine - INFO - Epoch(train)  [40][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:39:03  time: 0.2163  data_time: 0.0080  memory: 3357  loss: 4.4305  loss_cls: 0.6836  loss_bbox: 2.3528  loss_obj: 1.3942
03/19 20:35:06 - mmengine - INFO - Epoch(train)  [40][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:38:52  time: 0.2188  data_time: 0.0083  memory: 3937  loss: 4.3673  loss_cls: 0.6773  loss_bbox: 2.3315  loss_obj: 1.3584
03/19 20:35:16 - mmengine - INFO - Epoch(train)  [40][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:38:40  time: 0.1993  data_time: 0.0081  memory: 2587  loss: 4.4218  loss_cls: 0.6991  loss_bbox: 2.3653  loss_obj: 1.3574
03/19 20:35:26 - mmengine - INFO - Epoch(train)  [40][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:38:29  time: 0.2119  data_time: 0.0080  memory: 3071  loss: 4.3393  loss_cls: 0.6821  loss_bbox: 2.3202  loss_obj: 1.3370
03/19 20:35:36 - mmengine - INFO - Epoch(train)  [40][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:38:17  time: 0.1988  data_time: 0.0080  memory: 2817  loss: 4.4240  loss_cls: 0.6946  loss_bbox: 2.3649  loss_obj: 1.3645
03/19 20:35:46 - mmengine - INFO - Epoch(train)  [40][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:38:05  time: 0.2006  data_time: 0.0083  memory: 3639  loss: 4.3537  loss_cls: 0.6852  loss_bbox: 2.3445  loss_obj: 1.3239
03/19 20:35:58 - mmengine - INFO - Epoch(train)  [40][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:55  time: 0.2329  data_time: 0.0081  memory: 3937  loss: 4.3575  loss_cls: 0.6820  loss_bbox: 2.3210  loss_obj: 1.3545
03/19 20:36:09 - mmengine - INFO - Epoch(train)  [40][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:45  time: 0.2225  data_time: 0.0081  memory: 3937  loss: 4.4192  loss_cls: 0.6922  loss_bbox: 2.3431  loss_obj: 1.3839
03/19 20:36:21 - mmengine - INFO - Epoch(train)  [40][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:34  time: 0.2263  data_time: 0.0081  memory: 3937  loss: 4.4069  loss_cls: 0.6809  loss_bbox: 2.3305  loss_obj: 1.3955
03/19 20:36:32 - mmengine - INFO - Epoch(train)  [40][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:24  time: 0.2205  data_time: 0.0081  memory: 3357  loss: 4.4035  loss_cls: 0.6907  loss_bbox: 2.3525  loss_obj: 1.3604
03/19 20:36:43 - mmengine - INFO - Epoch(train)  [40][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:14  time: 0.2262  data_time: 0.0080  memory: 3639  loss: 4.4183  loss_cls: 0.6887  loss_bbox: 2.3322  loss_obj: 1.3974
03/19 20:36:54 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:36:54 - mmengine - INFO - Epoch(train)  [40][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:37:04  time: 0.2279  data_time: 0.0080  memory: 3937  loss: 4.3946  loss_cls: 0.6933  loss_bbox: 2.3453  loss_obj: 1.3559
03/19 20:36:54 - mmengine - INFO - Saving checkpoint at 40 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:36:59 - mmengine - INFO - Epoch(val)  [40][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 20:37:02 - mmengine - INFO - Epoch(val)  [40][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 20:37:05 - mmengine - INFO - Epoch(val)  [40][150/250]    eta: 0:00:05  time: 0.0604  data_time: 0.0091  memory: 527  
03/19 20:37:08 - mmengine - INFO - Epoch(val)  [40][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 20:37:11 - mmengine - INFO - Epoch(val)  [40][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0065  memory: 527  
03/19 20:37:13 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.81s).
Accumulating evaluation results...
DONE (t=2.12s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.211
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.506
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.262
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.252
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.492
03/19 20:37:23 - mmengine - INFO - bbox_mAP_copypaste: 0.211 0.506 0.135 0.134 0.262 0.393
03/19 20:37:23 - mmengine - INFO - Epoch(val) [40][250/250]    coco/bbox_mAP: 0.2110  coco/bbox_mAP_50: 0.5060  coco/bbox_mAP_75: 0.1350  coco/bbox_mAP_s: 0.1340  coco/bbox_mAP_m: 0.2620  coco/bbox_mAP_l: 0.3930  data_time: 0.0071  time: 0.0589
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:37:34 - mmengine - INFO - Epoch(train)  [41][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:53  time: 0.2195  data_time: 0.0190  memory: 3357  loss: 4.3992  loss_cls: 0.6915  loss_bbox: 2.3390  loss_obj: 1.3687
03/19 20:37:45 - mmengine - INFO - Epoch(train)  [41][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:42  time: 0.2146  data_time: 0.0081  memory: 3937  loss: 4.4367  loss_cls: 0.6935  loss_bbox: 2.3616  loss_obj: 1.3817
03/19 20:37:56 - mmengine - INFO - Epoch(train)  [41][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:31  time: 0.2201  data_time: 0.0080  memory: 3639  loss: 4.3684  loss_cls: 0.6784  loss_bbox: 2.3465  loss_obj: 1.3435
03/19 20:38:08 - mmengine - INFO - Epoch(train)  [41][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:22  time: 0.2448  data_time: 0.0080  memory: 3937  loss: 4.3852  loss_cls: 0.6754  loss_bbox: 2.3198  loss_obj: 1.3900
03/19 20:38:20 - mmengine - INFO - Epoch(train)  [41][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:13  time: 0.2328  data_time: 0.0081  memory: 3937  loss: 4.4108  loss_cls: 0.6840  loss_bbox: 2.3426  loss_obj: 1.3842
03/19 20:38:31 - mmengine - INFO - Epoch(train)  [41][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:36:02  time: 0.2130  data_time: 0.0082  memory: 3639  loss: 4.3560  loss_cls: 0.6812  loss_bbox: 2.3188  loss_obj: 1.3560
03/19 20:38:42 - mmengine - INFO - Epoch(train)  [41][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:35:52  time: 0.2300  data_time: 0.0083  memory: 3937  loss: 4.4323  loss_cls: 0.6913  loss_bbox: 2.3589  loss_obj: 1.3821
03/19 20:38:53 - mmengine - INFO - Epoch(train)  [41][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:35:41  time: 0.2183  data_time: 0.0081  memory: 3639  loss: 4.4402  loss_cls: 0.6897  loss_bbox: 2.3522  loss_obj: 1.3982
03/19 20:39:02 - mmengine - INFO - Epoch(train)  [41][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:35:28  time: 0.1879  data_time: 0.0082  memory: 2587  loss: 4.4449  loss_cls: 0.6999  loss_bbox: 2.3968  loss_obj: 1.3483
03/19 20:39:12 - mmengine - INFO - Epoch(train)  [41][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:35:15  time: 0.1888  data_time: 0.0083  memory: 3071  loss: 4.4109  loss_cls: 0.6951  loss_bbox: 2.3670  loss_obj: 1.3488
03/19 20:39:21 - mmengine - INFO - Epoch(train)  [41][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:35:02  time: 0.1908  data_time: 0.0082  memory: 2587  loss: 4.4216  loss_cls: 0.6942  loss_bbox: 2.3842  loss_obj: 1.3433
03/19 20:39:34 - mmengine - INFO - Epoch(train)  [41][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:53  time: 0.2433  data_time: 0.0081  memory: 3937  loss: 4.3790  loss_cls: 0.6775  loss_bbox: 2.3274  loss_obj: 1.3740
03/19 20:39:45 - mmengine - INFO - Epoch(train)  [41][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:43  time: 0.2234  data_time: 0.0080  memory: 3357  loss: 4.4328  loss_cls: 0.6819  loss_bbox: 2.3391  loss_obj: 1.4118
03/19 20:39:56 - mmengine - INFO - Epoch(train)  [41][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:32  time: 0.2204  data_time: 0.0080  memory: 3639  loss: 4.4287  loss_cls: 0.6920  loss_bbox: 2.3744  loss_obj: 1.3623
03/19 20:40:07 - mmengine - INFO - Epoch(train)  [41][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:22  time: 0.2209  data_time: 0.0082  memory: 3937  loss: 4.3935  loss_cls: 0.6877  loss_bbox: 2.3465  loss_obj: 1.3593
03/19 20:40:18 - mmengine - INFO - Epoch(train)  [41][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:11  time: 0.2218  data_time: 0.0081  memory: 3639  loss: 4.3709  loss_cls: 0.6889  loss_bbox: 2.3339  loss_obj: 1.3482
03/19 20:40:29 - mmengine - INFO - Epoch(train)  [41][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:34:00  time: 0.2172  data_time: 0.0082  memory: 3639  loss: 4.4090  loss_cls: 0.6917  loss_bbox: 2.3569  loss_obj: 1.3604
03/19 20:40:39 - mmengine - INFO - Epoch(train)  [41][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:33:48  time: 0.1996  data_time: 0.0083  memory: 3937  loss: 4.4413  loss_cls: 0.6987  loss_bbox: 2.3678  loss_obj: 1.3748
03/19 20:40:50 - mmengine - INFO - Epoch(train)  [41][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:33:37  time: 0.2124  data_time: 0.0082  memory: 3357  loss: 4.4194  loss_cls: 0.6904  loss_bbox: 2.3415  loss_obj: 1.3875
03/19 20:41:00 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:41:00 - mmengine - INFO - Epoch(train)  [41][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:33:26  time: 0.2162  data_time: 0.0081  memory: 3937  loss: 4.4491  loss_cls: 0.6901  loss_bbox: 2.3718  loss_obj: 1.3871
03/19 20:41:00 - mmengine - INFO - Saving checkpoint at 41 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:41:06 - mmengine - INFO - Epoch(val)  [41][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0070  memory: 527  
03/19 20:41:09 - mmengine - INFO - Epoch(val)  [41][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 20:41:12 - mmengine - INFO - Epoch(val)  [41][150/250]    eta: 0:00:05  time: 0.0594  data_time: 0.0065  memory: 527  
03/19 20:41:14 - mmengine - INFO - Epoch(val)  [41][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 20:41:17 - mmengine - INFO - Epoch(val)  [41][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0066  memory: 527  
03/19 20:41:19 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.92s).
Accumulating evaluation results...
DONE (t=2.14s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.212
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.509
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.136
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.394
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.315
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.253
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.496
03/19 20:41:29 - mmengine - INFO - bbox_mAP_copypaste: 0.212 0.509 0.136 0.135 0.264 0.394
03/19 20:41:30 - mmengine - INFO - Epoch(val) [41][250/250]    coco/bbox_mAP: 0.2120  coco/bbox_mAP_50: 0.5090  coco/bbox_mAP_75: 0.1360  coco/bbox_mAP_s: 0.1350  coco/bbox_mAP_m: 0.2640  coco/bbox_mAP_l: 0.3940  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:41:42 - mmengine - INFO - Epoch(train)  [42][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:33:18  time: 0.2550  data_time: 0.0185  memory: 3937  loss: 4.3859  loss_cls: 0.6826  loss_bbox: 2.3448  loss_obj: 1.3584
03/19 20:41:53 - mmengine - INFO - Epoch(train)  [42][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:33:07  time: 0.2156  data_time: 0.0081  memory: 3937  loss: 4.4855  loss_cls: 0.6924  loss_bbox: 2.3948  loss_obj: 1.3983
03/19 20:42:05 - mmengine - INFO - Epoch(train)  [42][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:57  time: 0.2345  data_time: 0.0081  memory: 3639  loss: 4.4586  loss_cls: 0.6935  loss_bbox: 2.3675  loss_obj: 1.3975
03/19 20:42:16 - mmengine - INFO - Epoch(train)  [42][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:47  time: 0.2207  data_time: 0.0081  memory: 3639  loss: 4.4071  loss_cls: 0.6852  loss_bbox: 2.3467  loss_obj: 1.3752
03/19 20:42:26 - mmengine - INFO - Epoch(train)  [42][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:35  time: 0.2090  data_time: 0.0083  memory: 3639  loss: 4.5160  loss_cls: 0.7031  loss_bbox: 2.4011  loss_obj: 1.4118
03/19 20:42:37 - mmengine - INFO - Epoch(train)  [42][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:24  time: 0.2130  data_time: 0.0081  memory: 2817  loss: 4.3940  loss_cls: 0.6863  loss_bbox: 2.3278  loss_obj: 1.3799
03/19 20:42:48 - mmengine - INFO - Epoch(train)  [42][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:13  time: 0.2180  data_time: 0.0082  memory: 3937  loss: 4.4248  loss_cls: 0.6986  loss_bbox: 2.3604  loss_obj: 1.3658
03/19 20:42:58 - mmengine - INFO - Epoch(train)  [42][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:32:01  time: 0.1995  data_time: 0.0083  memory: 3937  loss: 4.3990  loss_cls: 0.6843  loss_bbox: 2.3542  loss_obj: 1.3605
03/19 20:43:10 - mmengine - INFO - Epoch(train)  [42][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:31:52  time: 0.2385  data_time: 0.0081  memory: 3937  loss: 4.3420  loss_cls: 0.6777  loss_bbox: 2.3179  loss_obj: 1.3464
03/19 20:43:20 - mmengine - INFO - Epoch(train)  [42][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:31:40  time: 0.1976  data_time: 0.0080  memory: 2587  loss: 4.3910  loss_cls: 0.6991  loss_bbox: 2.3534  loss_obj: 1.3384
03/19 20:43:31 - mmengine - INFO - Epoch(train)  [42][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:31:29  time: 0.2187  data_time: 0.0080  memory: 3937  loss: 4.4149  loss_cls: 0.6900  loss_bbox: 2.3665  loss_obj: 1.3584
03/19 20:43:41 - mmengine - INFO - Epoch(train)  [42][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:31:17  time: 0.2044  data_time: 0.0080  memory: 3071  loss: 4.3674  loss_cls: 0.6855  loss_bbox: 2.3441  loss_obj: 1.3379
03/19 20:43:51 - mmengine - INFO - Epoch(train)  [42][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:31:06  time: 0.2063  data_time: 0.0082  memory: 3071  loss: 4.4035  loss_cls: 0.6875  loss_bbox: 2.3655  loss_obj: 1.3505
03/19 20:44:04 - mmengine - INFO - Epoch(train)  [42][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:57  time: 0.2518  data_time: 0.0080  memory: 3937  loss: 4.3857  loss_cls: 0.6875  loss_bbox: 2.3207  loss_obj: 1.3775
03/19 20:44:14 - mmengine - INFO - Epoch(train)  [42][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:45  time: 0.2001  data_time: 0.0082  memory: 3639  loss: 4.4062  loss_cls: 0.6852  loss_bbox: 2.3726  loss_obj: 1.3484
03/19 20:44:23 - mmengine - INFO - Epoch(train)  [42][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:33  time: 0.1919  data_time: 0.0082  memory: 3071  loss: 4.4122  loss_cls: 0.6939  loss_bbox: 2.3727  loss_obj: 1.3456
03/19 20:44:35 - mmengine - INFO - Epoch(train)  [42][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:22  time: 0.2218  data_time: 0.0082  memory: 3937  loss: 4.3871  loss_cls: 0.6812  loss_bbox: 2.3441  loss_obj: 1.3618
03/19 20:44:45 - mmengine - INFO - Epoch(train)  [42][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:10  time: 0.2031  data_time: 0.0082  memory: 3357  loss: 4.4101  loss_cls: 0.6948  loss_bbox: 2.3750  loss_obj: 1.3403
03/19 20:44:56 - mmengine - INFO - Epoch(train)  [42][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:30:00  time: 0.2199  data_time: 0.0081  memory: 3639  loss: 4.3985  loss_cls: 0.6807  loss_bbox: 2.3660  loss_obj: 1.3518
03/19 20:45:06 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:45:06 - mmengine - INFO - Epoch(train)  [42][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:29:48  time: 0.2040  data_time: 0.0081  memory: 3639  loss: 4.4784  loss_cls: 0.7002  loss_bbox: 2.3890  loss_obj: 1.3893
03/19 20:45:06 - mmengine - INFO - Saving checkpoint at 42 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:45:11 - mmengine - INFO - Epoch(val)  [42][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0072  memory: 527  
03/19 20:45:14 - mmengine - INFO - Epoch(val)  [42][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 20:45:17 - mmengine - INFO - Epoch(val)  [42][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 20:45:20 - mmengine - INFO - Epoch(val)  [42][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 20:45:23 - mmengine - INFO - Epoch(val)  [42][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0065  memory: 527  
03/19 20:45:24 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.65s).
Accumulating evaluation results...
DONE (t=2.12s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.213
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.512
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.133
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.266
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.390
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.384
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.494
03/19 20:45:35 - mmengine - INFO - bbox_mAP_copypaste: 0.213 0.512 0.133 0.135 0.266 0.390
03/19 20:45:35 - mmengine - INFO - Epoch(val) [42][250/250]    coco/bbox_mAP: 0.2130  coco/bbox_mAP_50: 0.5120  coco/bbox_mAP_75: 0.1330  coco/bbox_mAP_s: 0.1350  coco/bbox_mAP_m: 0.2660  coco/bbox_mAP_l: 0.3900  data_time: 0.0066  time: 0.0582
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:45:46 - mmengine - INFO - Epoch(train)  [43][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:29:38  time: 0.2303  data_time: 0.0185  memory: 3639  loss: 4.4176  loss_cls: 0.6872  loss_bbox: 2.3567  loss_obj: 1.3737
03/19 20:45:57 - mmengine - INFO - Epoch(train)  [43][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:29:26  time: 0.2095  data_time: 0.0081  memory: 3937  loss: 4.4579  loss_cls: 0.6995  loss_bbox: 2.3697  loss_obj: 1.3887
03/19 20:46:08 - mmengine - INFO - Epoch(train)  [43][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:29:16  time: 0.2184  data_time: 0.0083  memory: 3937  loss: 4.3717  loss_cls: 0.6781  loss_bbox: 2.3600  loss_obj: 1.3336
03/19 20:46:19 - mmengine - INFO - Epoch(train)  [43][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:29:05  time: 0.2209  data_time: 0.0080  memory: 3937  loss: 4.3102  loss_cls: 0.6696  loss_bbox: 2.3018  loss_obj: 1.3388
03/19 20:46:31 - mmengine - INFO - Epoch(train)  [43][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:56  time: 0.2419  data_time: 0.0080  memory: 3937  loss: 4.4290  loss_cls: 0.6912  loss_bbox: 2.3462  loss_obj: 1.3917
03/19 20:46:42 - mmengine - INFO - Epoch(train)  [43][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:45  time: 0.2190  data_time: 0.0081  memory: 3357  loss: 4.4238  loss_cls: 0.6855  loss_bbox: 2.3488  loss_obj: 1.3894
03/19 20:46:53 - mmengine - INFO - Epoch(train)  [43][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:35  time: 0.2209  data_time: 0.0082  memory: 3639  loss: 4.4123  loss_cls: 0.6800  loss_bbox: 2.3534  loss_obj: 1.3789
03/19 20:47:04 - mmengine - INFO - Epoch(train)  [43][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:23  time: 0.2125  data_time: 0.0083  memory: 3937  loss: 4.4677  loss_cls: 0.6966  loss_bbox: 2.3906  loss_obj: 1.3805
03/19 20:47:15 - mmengine - INFO - Epoch(train)  [43][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:14  time: 0.2300  data_time: 0.0081  memory: 3937  loss: 4.3508  loss_cls: 0.6789  loss_bbox: 2.3327  loss_obj: 1.3392
03/19 20:47:26 - mmengine - INFO - Epoch(train)  [43][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:28:02  time: 0.2102  data_time: 0.0082  memory: 3357  loss: 4.4328  loss_cls: 0.6927  loss_bbox: 2.3724  loss_obj: 1.3678
03/19 20:47:37 - mmengine - INFO - Epoch(train)  [43][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:27:52  time: 0.2249  data_time: 0.0082  memory: 3937  loss: 4.3517  loss_cls: 0.6763  loss_bbox: 2.3482  loss_obj: 1.3273
03/19 20:47:47 - mmengine - INFO - Epoch(train)  [43][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:27:40  time: 0.2069  data_time: 0.0083  memory: 3639  loss: 4.3776  loss_cls: 0.6823  loss_bbox: 2.3512  loss_obj: 1.3441
03/19 20:47:58 - mmengine - INFO - Epoch(train)  [43][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:27:29  time: 0.2153  data_time: 0.0082  memory: 3937  loss: 4.3994  loss_cls: 0.6851  loss_bbox: 2.3673  loss_obj: 1.3471
03/19 20:48:09 - mmengine - INFO - Epoch(train)  [43][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:27:18  time: 0.2101  data_time: 0.0082  memory: 3639  loss: 4.4109  loss_cls: 0.6817  loss_bbox: 2.3595  loss_obj: 1.3697
03/19 20:48:20 - mmengine - INFO - Epoch(train)  [43][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:27:08  time: 0.2267  data_time: 0.0083  memory: 3937  loss: 4.4538  loss_cls: 0.6888  loss_bbox: 2.3766  loss_obj: 1.3884
03/19 20:48:31 - mmengine - INFO - Epoch(train)  [43][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:57  time: 0.2128  data_time: 0.0082  memory: 3357  loss: 4.3712  loss_cls: 0.6755  loss_bbox: 2.3490  loss_obj: 1.3467
03/19 20:48:43 - mmengine - INFO - Epoch(train)  [43][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:47  time: 0.2404  data_time: 0.0083  memory: 3937  loss: 4.3890  loss_cls: 0.6789  loss_bbox: 2.3274  loss_obj: 1.3827
03/19 20:48:55 - mmengine - INFO - Epoch(train)  [43][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:39  time: 0.2520  data_time: 0.0081  memory: 3937  loss: 4.3188  loss_cls: 0.6719  loss_bbox: 2.3073  loss_obj: 1.3395
03/19 20:49:08 - mmengine - INFO - Epoch(train)  [43][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:31  time: 0.2554  data_time: 0.0082  memory: 3937  loss: 4.5329  loss_cls: 0.6959  loss_bbox: 2.3776  loss_obj: 1.4594
03/19 20:49:18 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:49:18 - mmengine - INFO - Epoch(train)  [43][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:19  time: 0.2003  data_time: 0.0082  memory: 3639  loss: 4.4342  loss_cls: 0.6914  loss_bbox: 2.3899  loss_obj: 1.3529
03/19 20:49:18 - mmengine - INFO - Saving checkpoint at 43 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:49:23 - mmengine - INFO - Epoch(val)  [43][ 50/250]    eta: 0:00:11  time: 0.0589  data_time: 0.0073  memory: 527  
03/19 20:49:26 - mmengine - INFO - Epoch(val)  [43][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 20:49:29 - mmengine - INFO - Epoch(val)  [43][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0064  memory: 527  
03/19 20:49:32 - mmengine - INFO - Epoch(val)  [43][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 20:49:35 - mmengine - INFO - Epoch(val)  [43][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0064  memory: 527  
03/19 20:49:36 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.28s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.88s).
Accumulating evaluation results...
DONE (t=2.13s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.214
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.515
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.133
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.266
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.494
03/19 20:49:47 - mmengine - INFO - bbox_mAP_copypaste: 0.214 0.515 0.133 0.135 0.266 0.391
03/19 20:49:47 - mmengine - INFO - Epoch(val) [43][250/250]    coco/bbox_mAP: 0.2140  coco/bbox_mAP_50: 0.5150  coco/bbox_mAP_75: 0.1330  coco/bbox_mAP_s: 0.1350  coco/bbox_mAP_m: 0.2660  coco/bbox_mAP_l: 0.3910  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:49:59 - mmengine - INFO - Epoch(train)  [44][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:26:09  time: 0.2351  data_time: 0.0189  memory: 3937  loss: 4.3644  loss_cls: 0.6800  loss_bbox: 2.3334  loss_obj: 1.3510
03/19 20:50:10 - mmengine - INFO - Epoch(train)  [44][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:58  time: 0.2132  data_time: 0.0081  memory: 3937  loss: 4.4057  loss_cls: 0.6775  loss_bbox: 2.3536  loss_obj: 1.3746
03/19 20:50:20 - mmengine - INFO - Epoch(train)  [44][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:46  time: 0.2046  data_time: 0.0082  memory: 3071  loss: 4.4037  loss_cls: 0.6886  loss_bbox: 2.3550  loss_obj: 1.3601
03/19 20:50:31 - mmengine - INFO - Epoch(train)  [44][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:35  time: 0.2122  data_time: 0.0083  memory: 3357  loss: 4.3727  loss_cls: 0.6774  loss_bbox: 2.3202  loss_obj: 1.3751
03/19 20:50:41 - mmengine - INFO - Epoch(train)  [44][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:24  time: 0.2096  data_time: 0.0081  memory: 3071  loss: 4.3839  loss_cls: 0.6855  loss_bbox: 2.3617  loss_obj: 1.3367
03/19 20:50:52 - mmengine - INFO - Epoch(train)  [44][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:12  time: 0.2120  data_time: 0.0082  memory: 3937  loss: 4.3886  loss_cls: 0.6907  loss_bbox: 2.3328  loss_obj: 1.3651
03/19 20:51:02 - mmengine - INFO - Epoch(train)  [44][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:25:01  time: 0.2069  data_time: 0.0081  memory: 3357  loss: 4.3987  loss_cls: 0.6783  loss_bbox: 2.3541  loss_obj: 1.3663
03/19 20:51:13 - mmengine - INFO - Epoch(train)  [44][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:24:50  time: 0.2246  data_time: 0.0081  memory: 3937  loss: 4.3839  loss_cls: 0.6751  loss_bbox: 2.3329  loss_obj: 1.3759
03/19 20:51:23 - mmengine - INFO - Epoch(train)  [44][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:24:38  time: 0.1934  data_time: 0.0082  memory: 2587  loss: 4.4274  loss_cls: 0.6953  loss_bbox: 2.3565  loss_obj: 1.3756
03/19 20:51:34 - mmengine - INFO - Epoch(train)  [44][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:24:27  time: 0.2210  data_time: 0.0082  memory: 3357  loss: 4.4276  loss_cls: 0.6945  loss_bbox: 2.3730  loss_obj: 1.3600
03/19 20:51:43 - mmengine - INFO - Epoch(train)  [44][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:24:14  time: 0.1820  data_time: 0.0084  memory: 2817  loss: 4.4662  loss_cls: 0.6909  loss_bbox: 2.4179  loss_obj: 1.3574
03/19 20:51:54 - mmengine - INFO - Epoch(train)  [44][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:24:03  time: 0.2089  data_time: 0.0085  memory: 3937  loss: 4.3185  loss_cls: 0.6796  loss_bbox: 2.3216  loss_obj: 1.3173
03/19 20:52:05 - mmengine - INFO - Epoch(train)  [44][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:52  time: 0.2186  data_time: 0.0084  memory: 3937  loss: 4.4706  loss_cls: 0.6890  loss_bbox: 2.3778  loss_obj: 1.4038
03/19 20:52:16 - mmengine - INFO - Epoch(train)  [44][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:42  time: 0.2290  data_time: 0.0082  memory: 3937  loss: 4.4218  loss_cls: 0.6874  loss_bbox: 2.3651  loss_obj: 1.3693
03/19 20:52:27 - mmengine - INFO - Epoch(train)  [44][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:32  time: 0.2270  data_time: 0.0083  memory: 3937  loss: 4.3957  loss_cls: 0.6895  loss_bbox: 2.3581  loss_obj: 1.3481
03/19 20:52:39 - mmengine - INFO - Epoch(train)  [44][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:21  time: 0.2213  data_time: 0.0082  memory: 3937  loss: 4.4659  loss_cls: 0.6969  loss_bbox: 2.3615  loss_obj: 1.4074
03/19 20:52:49 - mmengine - INFO - Epoch(train)  [44][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:10  time: 0.2155  data_time: 0.0083  memory: 3937  loss: 4.4095  loss_cls: 0.6827  loss_bbox: 2.3480  loss_obj: 1.3788
03/19 20:53:00 - mmengine - INFO - Epoch(train)  [44][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:23:00  time: 0.2227  data_time: 0.0082  memory: 3639  loss: 4.5228  loss_cls: 0.6950  loss_bbox: 2.3903  loss_obj: 1.4376
03/19 20:53:11 - mmengine - INFO - Epoch(train)  [44][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:22:48  time: 0.2096  data_time: 0.0082  memory: 3357  loss: 4.3714  loss_cls: 0.6815  loss_bbox: 2.3316  loss_obj: 1.3582
03/19 20:53:22 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:53:22 - mmengine - INFO - Epoch(train)  [44][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:22:38  time: 0.2158  data_time: 0.0081  memory: 3639  loss: 4.3726  loss_cls: 0.6744  loss_bbox: 2.3426  loss_obj: 1.3555
03/19 20:53:22 - mmengine - INFO - Saving checkpoint at 44 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:53:27 - mmengine - INFO - Epoch(val)  [44][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0071  memory: 527  
03/19 20:53:30 - mmengine - INFO - Epoch(val)  [44][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0064  memory: 527  
03/19 20:53:33 - mmengine - INFO - Epoch(val)  [44][150/250]    eta: 0:00:05  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 20:53:36 - mmengine - INFO - Epoch(val)  [44][200/250]    eta: 0:00:02  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 20:53:39 - mmengine - INFO - Epoch(val)  [44][250/250]    eta: 0:00:00  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 20:53:40 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.88s).
Accumulating evaluation results...
DONE (t=2.11s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.214
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.515
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.135
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.497
03/19 20:53:51 - mmengine - INFO - bbox_mAP_copypaste: 0.214 0.515 0.132 0.135 0.264 0.393
03/19 20:53:51 - mmengine - INFO - Epoch(val) [44][250/250]    coco/bbox_mAP: 0.2140  coco/bbox_mAP_50: 0.5150  coco/bbox_mAP_75: 0.1320  coco/bbox_mAP_s: 0.1350  coco/bbox_mAP_m: 0.2640  coco/bbox_mAP_l: 0.3930  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:54:01 - mmengine - INFO - Epoch(train)  [45][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:22:26  time: 0.2116  data_time: 0.0185  memory: 3357  loss: 4.3593  loss_cls: 0.6907  loss_bbox: 2.3348  loss_obj: 1.3338
03/19 20:54:11 - mmengine - INFO - Epoch(train)  [45][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:22:14  time: 0.1948  data_time: 0.0084  memory: 3071  loss: 4.4565  loss_cls: 0.6889  loss_bbox: 2.3887  loss_obj: 1.3789
03/19 20:54:22 - mmengine - INFO - Epoch(train)  [45][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:22:03  time: 0.2216  data_time: 0.0081  memory: 3357  loss: 4.4068  loss_cls: 0.6831  loss_bbox: 2.3710  loss_obj: 1.3527
03/19 20:54:33 - mmengine - INFO - Epoch(train)  [45][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:21:52  time: 0.2100  data_time: 0.0080  memory: 3937  loss: 4.4016  loss_cls: 0.6884  loss_bbox: 2.3672  loss_obj: 1.3460
03/19 20:54:43 - mmengine - INFO - Epoch(train)  [45][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:21:41  time: 0.2101  data_time: 0.0082  memory: 3357  loss: 4.4354  loss_cls: 0.6861  loss_bbox: 2.3733  loss_obj: 1.3761
03/19 20:54:53 - mmengine - INFO - Epoch(train)  [45][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:21:29  time: 0.1962  data_time: 0.0084  memory: 3071  loss: 4.4219  loss_cls: 0.6923  loss_bbox: 2.3700  loss_obj: 1.3595
03/19 20:55:03 - mmengine - INFO - Epoch(train)  [45][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:21:17  time: 0.1988  data_time: 0.0083  memory: 3639  loss: 4.4329  loss_cls: 0.6927  loss_bbox: 2.3907  loss_obj: 1.3495
03/19 20:55:12 - mmengine - INFO - Epoch(train)  [45][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:21:04  time: 0.1848  data_time: 0.0085  memory: 2587  loss: 4.4696  loss_cls: 0.6959  loss_bbox: 2.3972  loss_obj: 1.3764
03/19 20:55:23 - mmengine - INFO - Epoch(train)  [45][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:20:52  time: 0.2022  data_time: 0.0081  memory: 3357  loss: 4.4579  loss_cls: 0.6981  loss_bbox: 2.3793  loss_obj: 1.3805
03/19 20:55:33 - mmengine - INFO - Epoch(train)  [45][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:20:41  time: 0.2115  data_time: 0.0080  memory: 3071  loss: 4.3489  loss_cls: 0.6739  loss_bbox: 2.3381  loss_obj: 1.3368
03/19 20:55:43 - mmengine - INFO - Epoch(train)  [45][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:20:29  time: 0.2068  data_time: 0.0081  memory: 3357  loss: 4.3880  loss_cls: 0.6860  loss_bbox: 2.3412  loss_obj: 1.3609
03/19 20:55:55 - mmengine - INFO - Epoch(train)  [45][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:20:19  time: 0.2226  data_time: 0.0082  memory: 3639  loss: 4.4119  loss_cls: 0.6876  loss_bbox: 2.3525  loss_obj: 1.3719
03/19 20:56:05 - mmengine - INFO - Epoch(train)  [45][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:20:08  time: 0.2141  data_time: 0.0081  memory: 3639  loss: 4.3798  loss_cls: 0.6846  loss_bbox: 2.3487  loss_obj: 1.3465
03/19 20:56:17 - mmengine - INFO - Epoch(train)  [45][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:58  time: 0.2323  data_time: 0.0081  memory: 3639  loss: 4.3277  loss_cls: 0.6775  loss_bbox: 2.3138  loss_obj: 1.3364
03/19 20:56:27 - mmengine - INFO - Epoch(train)  [45][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:47  time: 0.2086  data_time: 0.0082  memory: 3639  loss: 4.3938  loss_cls: 0.6855  loss_bbox: 2.3852  loss_obj: 1.3232
03/19 20:56:37 - mmengine - INFO - Epoch(train)  [45][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:34  time: 0.1939  data_time: 0.0082  memory: 2587  loss: 4.4049  loss_cls: 0.6939  loss_bbox: 2.3736  loss_obj: 1.3374
03/19 20:56:48 - mmengine - INFO - Epoch(train)  [45][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:23  time: 0.2122  data_time: 0.0082  memory: 3937  loss: 4.3787  loss_cls: 0.6820  loss_bbox: 2.3832  loss_obj: 1.3134
03/19 20:56:58 - mmengine - INFO - Epoch(train)  [45][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:11  time: 0.1991  data_time: 0.0080  memory: 2587  loss: 4.3745  loss_cls: 0.6827  loss_bbox: 2.3520  loss_obj: 1.3398
03/19 20:57:10 - mmengine - INFO - Epoch(train)  [45][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:19:02  time: 0.2444  data_time: 0.0080  memory: 3937  loss: 4.3711  loss_cls: 0.6779  loss_bbox: 2.2981  loss_obj: 1.3951
03/19 20:57:19 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 20:57:19 - mmengine - INFO - Epoch(train)  [45][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:18:49  time: 0.1844  data_time: 0.0082  memory: 3071  loss: 4.3714  loss_cls: 0.6840  loss_bbox: 2.3522  loss_obj: 1.3352
03/19 20:57:19 - mmengine - INFO - Saving checkpoint at 45 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:57:25 - mmengine - INFO - Epoch(val)  [45][ 50/250]    eta: 0:00:12  time: 0.0601  data_time: 0.0072  memory: 527  
03/19 20:57:27 - mmengine - INFO - Epoch(val)  [45][100/250]    eta: 0:00:08  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 20:57:30 - mmengine - INFO - Epoch(val)  [45][150/250]    eta: 0:00:05  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 20:57:33 - mmengine - INFO - Epoch(val)  [45][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0064  memory: 527  
03/19 20:57:36 - mmengine - INFO - Epoch(val)  [45][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0064  memory: 527  
03/19 20:57:38 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.10s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.55s).
Accumulating evaluation results...
DONE (t=2.09s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.216
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.522
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.134
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.268
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.401
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.320
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.320
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.320
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.259
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.503
03/19 20:57:48 - mmengine - INFO - bbox_mAP_copypaste: 0.216 0.522 0.134 0.137 0.268 0.401
03/19 20:57:48 - mmengine - INFO - Epoch(val) [45][250/250]    coco/bbox_mAP: 0.2160  coco/bbox_mAP_50: 0.5220  coco/bbox_mAP_75: 0.1340  coco/bbox_mAP_s: 0.1370  coco/bbox_mAP_m: 0.2680  coco/bbox_mAP_l: 0.4010  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 20:58:00 - mmengine - INFO - Epoch(train)  [46][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:18:39  time: 0.2320  data_time: 0.0187  memory: 3937  loss: 4.4067  loss_cls: 0.6906  loss_bbox: 2.3518  loss_obj: 1.3643
03/19 20:58:10 - mmengine - INFO - Epoch(train)  [46][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:18:28  time: 0.2153  data_time: 0.0081  memory: 3639  loss: 4.3953  loss_cls: 0.6878  loss_bbox: 2.3579  loss_obj: 1.3495
03/19 20:58:21 - mmengine - INFO - Epoch(train)  [46][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:18:17  time: 0.2048  data_time: 0.0081  memory: 3639  loss: 4.4792  loss_cls: 0.6991  loss_bbox: 2.4018  loss_obj: 1.3782
03/19 20:58:32 - mmengine - INFO - Epoch(train)  [46][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:18:07  time: 0.2299  data_time: 0.0082  memory: 3937  loss: 4.4148  loss_cls: 0.6929  loss_bbox: 2.3559  loss_obj: 1.3660
03/19 20:58:43 - mmengine - INFO - Epoch(train)  [46][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:55  time: 0.2062  data_time: 0.0082  memory: 3357  loss: 4.3756  loss_cls: 0.6875  loss_bbox: 2.3558  loss_obj: 1.3322
03/19 20:58:53 - mmengine - INFO - Epoch(train)  [46][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:44  time: 0.2143  data_time: 0.0081  memory: 3071  loss: 4.3554  loss_cls: 0.6812  loss_bbox: 2.3358  loss_obj: 1.3384
03/19 20:59:04 - mmengine - INFO - Epoch(train)  [46][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:33  time: 0.2048  data_time: 0.0083  memory: 3357  loss: 4.4087  loss_cls: 0.6854  loss_bbox: 2.3658  loss_obj: 1.3574
03/19 20:59:13 - mmengine - INFO - Epoch(train)  [46][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:20  time: 0.1914  data_time: 0.0082  memory: 2337  loss: 4.4270  loss_cls: 0.7021  loss_bbox: 2.3856  loss_obj: 1.3393
03/19 20:59:24 - mmengine - INFO - Epoch(train)  [46][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:10  time: 0.2256  data_time: 0.0081  memory: 3639  loss: 4.3035  loss_cls: 0.6690  loss_bbox: 2.3154  loss_obj: 1.3192
03/19 20:59:36 - mmengine - INFO - Epoch(train)  [46][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:17:00  time: 0.2294  data_time: 0.0082  memory: 3639  loss: 4.4125  loss_cls: 0.6817  loss_bbox: 2.3578  loss_obj: 1.3730
03/19 20:59:46 - mmengine - INFO - Epoch(train)  [46][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:16:48  time: 0.1989  data_time: 0.0081  memory: 2587  loss: 4.3531  loss_cls: 0.6925  loss_bbox: 2.3591  loss_obj: 1.3015
03/19 20:59:58 - mmengine - INFO - Epoch(train)  [46][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:16:38  time: 0.2388  data_time: 0.0081  memory: 3937  loss: 4.3545  loss_cls: 0.6726  loss_bbox: 2.3314  loss_obj: 1.3506
03/19 21:00:09 - mmengine - INFO - Epoch(train)  [46][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:16:28  time: 0.2280  data_time: 0.0082  memory: 3937  loss: 4.3720  loss_cls: 0.6773  loss_bbox: 2.3512  loss_obj: 1.3435
03/19 21:00:21 - mmengine - INFO - Epoch(train)  [46][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:16:18  time: 0.2257  data_time: 0.0082  memory: 3639  loss: 4.4225  loss_cls: 0.6846  loss_bbox: 2.3479  loss_obj: 1.3900
03/19 21:00:33 - mmengine - INFO - Epoch(train)  [46][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:16:09  time: 0.2417  data_time: 0.0081  memory: 3937  loss: 4.3354  loss_cls: 0.6637  loss_bbox: 2.3060  loss_obj: 1.3657
03/19 21:00:44 - mmengine - INFO - Epoch(train)  [46][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:58  time: 0.2247  data_time: 0.0081  memory: 3937  loss: 4.4019  loss_cls: 0.6848  loss_bbox: 2.3538  loss_obj: 1.3633
03/19 21:00:54 - mmengine - INFO - Epoch(train)  [46][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:47  time: 0.2061  data_time: 0.0081  memory: 3071  loss: 4.4085  loss_cls: 0.6953  loss_bbox: 2.3552  loss_obj: 1.3579
03/19 21:01:05 - mmengine - INFO - Epoch(train)  [46][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:36  time: 0.2238  data_time: 0.0084  memory: 3937  loss: 4.4228  loss_cls: 0.6851  loss_bbox: 2.3612  loss_obj: 1.3764
03/19 21:01:16 - mmengine - INFO - Epoch(train)  [46][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:25  time: 0.2094  data_time: 0.0080  memory: 2817  loss: 4.3518  loss_cls: 0.6844  loss_bbox: 2.3306  loss_obj: 1.3367
03/19 21:01:25 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:01:25 - mmengine - INFO - Epoch(train)  [46][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:12  time: 0.1743  data_time: 0.0081  memory: 1907  loss: 4.4095  loss_cls: 0.6932  loss_bbox: 2.3947  loss_obj: 1.3216
03/19 21:01:25 - mmengine - INFO - Saving checkpoint at 46 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:01:30 - mmengine - INFO - Epoch(val)  [46][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0073  memory: 527  
03/19 21:01:33 - mmengine - INFO - Epoch(val)  [46][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 21:01:36 - mmengine - INFO - Epoch(val)  [46][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 21:01:39 - mmengine - INFO - Epoch(val)  [46][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 21:01:42 - mmengine - INFO - Epoch(val)  [46][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0065  memory: 527  
03/19 21:01:43 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.54s).
Accumulating evaluation results...
DONE (t=2.08s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.218
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.525
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.270
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.422
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.260
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.506
03/19 21:01:53 - mmengine - INFO - bbox_mAP_copypaste: 0.218 0.525 0.137 0.138 0.270 0.422
03/19 21:01:53 - mmengine - INFO - Epoch(val) [46][250/250]    coco/bbox_mAP: 0.2180  coco/bbox_mAP_50: 0.5250  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1380  coco/bbox_mAP_m: 0.2700  coco/bbox_mAP_l: 0.4220  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:02:05 - mmengine - INFO - Epoch(train)  [47][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:15:01  time: 0.2252  data_time: 0.0183  memory: 3357  loss: 4.3851  loss_cls: 0.6799  loss_bbox: 2.3674  loss_obj: 1.3378
03/19 21:02:15 - mmengine - INFO - Epoch(train)  [47][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:14:50  time: 0.2087  data_time: 0.0082  memory: 3357  loss: 4.4303  loss_cls: 0.6853  loss_bbox: 2.3756  loss_obj: 1.3695
03/19 21:02:26 - mmengine - INFO - Epoch(train)  [47][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:14:39  time: 0.2076  data_time: 0.0083  memory: 3937  loss: 4.4416  loss_cls: 0.6893  loss_bbox: 2.3705  loss_obj: 1.3818
03/19 21:02:37 - mmengine - INFO - Epoch(train)  [47][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:14:28  time: 0.2210  data_time: 0.0081  memory: 3357  loss: 4.3499  loss_cls: 0.6745  loss_bbox: 2.3545  loss_obj: 1.3209
03/19 21:02:47 - mmengine - INFO - Epoch(train)  [47][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:14:16  time: 0.2062  data_time: 0.0082  memory: 3357  loss: 4.4152  loss_cls: 0.6826  loss_bbox: 2.3573  loss_obj: 1.3752
03/19 21:02:58 - mmengine - INFO - Epoch(train)  [47][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:14:06  time: 0.2206  data_time: 0.0082  memory: 3639  loss: 4.4139  loss_cls: 0.6919  loss_bbox: 2.3579  loss_obj: 1.3640
03/19 21:03:08 - mmengine - INFO - Epoch(train)  [47][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:13:54  time: 0.1938  data_time: 0.0080  memory: 2337  loss: 4.4013  loss_cls: 0.6837  loss_bbox: 2.3669  loss_obj: 1.3508
03/19 21:03:18 - mmengine - INFO - Epoch(train)  [47][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:13:42  time: 0.2041  data_time: 0.0081  memory: 2817  loss: 4.2890  loss_cls: 0.6577  loss_bbox: 2.2943  loss_obj: 1.3371
03/19 21:03:28 - mmengine - INFO - Epoch(train)  [47][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:13:31  time: 0.2063  data_time: 0.0082  memory: 3937  loss: 4.4283  loss_cls: 0.6913  loss_bbox: 2.3488  loss_obj: 1.3882
03/19 21:03:39 - mmengine - INFO - Epoch(train)  [47][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:13:20  time: 0.2153  data_time: 0.0082  memory: 3639  loss: 4.4204  loss_cls: 0.7029  loss_bbox: 2.3747  loss_obj: 1.3428
03/19 21:03:50 - mmengine - INFO - Epoch(train)  [47][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:13:09  time: 0.2220  data_time: 0.0083  memory: 3937  loss: 4.4163  loss_cls: 0.6813  loss_bbox: 2.3433  loss_obj: 1.3918
03/19 21:04:02 - mmengine - INFO - Epoch(train)  [47][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:59  time: 0.2291  data_time: 0.0081  memory: 3937  loss: 4.3670  loss_cls: 0.6862  loss_bbox: 2.3257  loss_obj: 1.3552
03/19 21:04:13 - mmengine - INFO - Epoch(train)  [47][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:49  time: 0.2296  data_time: 0.0081  memory: 3937  loss: 4.4225  loss_cls: 0.6836  loss_bbox: 2.3429  loss_obj: 1.3961
03/19 21:04:24 - mmengine - INFO - Epoch(train)  [47][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:38  time: 0.2236  data_time: 0.0083  memory: 3937  loss: 4.3712  loss_cls: 0.6820  loss_bbox: 2.3447  loss_obj: 1.3445
03/19 21:04:35 - mmengine - INFO - Epoch(train)  [47][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:28  time: 0.2168  data_time: 0.0082  memory: 3071  loss: 4.4604  loss_cls: 0.6885  loss_bbox: 2.3811  loss_obj: 1.3908
03/19 21:04:45 - mmengine - INFO - Epoch(train)  [47][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:16  time: 0.2035  data_time: 0.0081  memory: 3071  loss: 4.3959  loss_cls: 0.6825  loss_bbox: 2.3662  loss_obj: 1.3472
03/19 21:04:57 - mmengine - INFO - Epoch(train)  [47][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:12:06  time: 0.2283  data_time: 0.0083  memory: 3937  loss: 4.3931  loss_cls: 0.6825  loss_bbox: 2.3394  loss_obj: 1.3712
03/19 21:05:07 - mmengine - INFO - Epoch(train)  [47][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:54  time: 0.1999  data_time: 0.0082  memory: 3071  loss: 4.3872  loss_cls: 0.6834  loss_bbox: 2.3407  loss_obj: 1.3631
03/19 21:05:18 - mmengine - INFO - Epoch(train)  [47][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:44  time: 0.2276  data_time: 0.0080  memory: 3639  loss: 4.4020  loss_cls: 0.6893  loss_bbox: 2.3487  loss_obj: 1.3640
03/19 21:05:29 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:05:29 - mmengine - INFO - Epoch(train)  [47][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:33  time: 0.2219  data_time: 0.0082  memory: 3937  loss: 4.4172  loss_cls: 0.6883  loss_bbox: 2.3417  loss_obj: 1.3873
03/19 21:05:29 - mmengine - INFO - Saving checkpoint at 47 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:05:35 - mmengine - INFO - Epoch(val)  [47][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/19 21:05:37 - mmengine - INFO - Epoch(val)  [47][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 21:05:40 - mmengine - INFO - Epoch(val)  [47][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0066  memory: 527  
03/19 21:05:43 - mmengine - INFO - Epoch(val)  [47][200/250]    eta: 0:00:02  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 21:05:46 - mmengine - INFO - Epoch(val)  [47][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 21:05:48 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.52s).
Accumulating evaluation results...
DONE (t=2.30s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.218
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.527
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.139
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.270
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.423
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.261
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.512
03/19 21:05:58 - mmengine - INFO - bbox_mAP_copypaste: 0.218 0.527 0.138 0.139 0.270 0.423
03/19 21:05:58 - mmengine - INFO - Epoch(val) [47][250/250]    coco/bbox_mAP: 0.2180  coco/bbox_mAP_50: 0.5270  coco/bbox_mAP_75: 0.1380  coco/bbox_mAP_s: 0.1390  coco/bbox_mAP_m: 0.2700  coco/bbox_mAP_l: 0.4230  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:06:09 - mmengine - INFO - Epoch(train)  [48][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:23  time: 0.2197  data_time: 0.0188  memory: 3937  loss: 4.3648  loss_cls: 0.6872  loss_bbox: 2.3606  loss_obj: 1.3170
03/19 21:06:21 - mmengine - INFO - Epoch(train)  [48][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:13  time: 0.2381  data_time: 0.0082  memory: 3937  loss: 4.3812  loss_cls: 0.6888  loss_bbox: 2.3389  loss_obj: 1.3535
03/19 21:06:31 - mmengine - INFO - Epoch(train)  [48][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:11:01  time: 0.2045  data_time: 0.0081  memory: 2587  loss: 4.3661  loss_cls: 0.6978  loss_bbox: 2.3490  loss_obj: 1.3193
03/19 21:06:41 - mmengine - INFO - Epoch(train)  [48][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:10:49  time: 0.1887  data_time: 0.0082  memory: 2587  loss: 4.3988  loss_cls: 0.6996  loss_bbox: 2.3661  loss_obj: 1.3331
03/19 21:06:52 - mmengine - INFO - Epoch(train)  [48][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:10:39  time: 0.2291  data_time: 0.0082  memory: 3357  loss: 4.3766  loss_cls: 0.6794  loss_bbox: 2.3374  loss_obj: 1.3598
03/19 21:07:04 - mmengine - INFO - Epoch(train)  [48][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:10:28  time: 0.2256  data_time: 0.0081  memory: 3639  loss: 4.3680  loss_cls: 0.6842  loss_bbox: 2.3297  loss_obj: 1.3540
03/19 21:07:14 - mmengine - INFO - Epoch(train)  [48][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:10:17  time: 0.2133  data_time: 0.0080  memory: 3071  loss: 4.3823  loss_cls: 0.6879  loss_bbox: 2.3271  loss_obj: 1.3673
03/19 21:07:25 - mmengine - INFO - Epoch(train)  [48][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:10:06  time: 0.2054  data_time: 0.0082  memory: 3639  loss: 4.4388  loss_cls: 0.6906  loss_bbox: 2.3808  loss_obj: 1.3674
03/19 21:07:36 - mmengine - INFO - Epoch(train)  [48][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:56  time: 0.2266  data_time: 0.0080  memory: 3357  loss: 4.3816  loss_cls: 0.6820  loss_bbox: 2.3274  loss_obj: 1.3722
03/19 21:07:46 - mmengine - INFO - Epoch(train)  [48][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:44  time: 0.2116  data_time: 0.0082  memory: 3639  loss: 4.3826  loss_cls: 0.6780  loss_bbox: 2.3440  loss_obj: 1.3606
03/19 21:07:58 - mmengine - INFO - Epoch(train)  [48][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:35  time: 0.2373  data_time: 0.0082  memory: 3639  loss: 4.4075  loss_cls: 0.6787  loss_bbox: 2.3481  loss_obj: 1.3807
03/19 21:08:08 - mmengine - INFO - Epoch(train)  [48][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:23  time: 0.1931  data_time: 0.0081  memory: 3071  loss: 4.4000  loss_cls: 0.6914  loss_bbox: 2.3570  loss_obj: 1.3515
03/19 21:08:20 - mmengine - INFO - Epoch(train)  [48][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:13  time: 0.2409  data_time: 0.0081  memory: 3937  loss: 4.4123  loss_cls: 0.6734  loss_bbox: 2.3541  loss_obj: 1.3849
03/19 21:08:31 - mmengine - INFO - Epoch(train)  [48][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:09:02  time: 0.2112  data_time: 0.0082  memory: 3357  loss: 4.3274  loss_cls: 0.6787  loss_bbox: 2.3238  loss_obj: 1.3249
03/19 21:08:41 - mmengine - INFO - Epoch(train)  [48][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:08:51  time: 0.2141  data_time: 0.0083  memory: 3937  loss: 4.4133  loss_cls: 0.7011  loss_bbox: 2.3494  loss_obj: 1.3628
03/19 21:08:52 - mmengine - INFO - Epoch(train)  [48][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:08:40  time: 0.2146  data_time: 0.0082  memory: 3639  loss: 4.3546  loss_cls: 0.6798  loss_bbox: 2.3289  loss_obj: 1.3459
03/19 21:09:03 - mmengine - INFO - Epoch(train)  [48][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:08:29  time: 0.2113  data_time: 0.0082  memory: 3639  loss: 4.3976  loss_cls: 0.6796  loss_bbox: 2.3518  loss_obj: 1.3662
03/19 21:09:14 - mmengine - INFO - Epoch(train)  [48][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:08:18  time: 0.2232  data_time: 0.0081  memory: 3937  loss: 4.4065  loss_cls: 0.6864  loss_bbox: 2.3589  loss_obj: 1.3612
03/19 21:09:25 - mmengine - INFO - Epoch(train)  [48][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:08:08  time: 0.2230  data_time: 0.0082  memory: 3357  loss: 4.4114  loss_cls: 0.6818  loss_bbox: 2.3259  loss_obj: 1.4037
03/19 21:09:34 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:09:34 - mmengine - INFO - Epoch(train)  [48][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:55  time: 0.1838  data_time: 0.0083  memory: 3357  loss: 4.4492  loss_cls: 0.7005  loss_bbox: 2.4007  loss_obj: 1.3481
03/19 21:09:34 - mmengine - INFO - Saving checkpoint at 48 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:09:40 - mmengine - INFO - Epoch(val)  [48][ 50/250]    eta: 0:00:12  time: 0.0606  data_time: 0.0073  memory: 527  
03/19 21:09:42 - mmengine - INFO - Epoch(val)  [48][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 21:09:45 - mmengine - INFO - Epoch(val)  [48][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 21:09:48 - mmengine - INFO - Epoch(val)  [48][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 21:09:51 - mmengine - INFO - Epoch(val)  [48][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0064  memory: 527  
03/19 21:09:53 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.66s).
Accumulating evaluation results...
DONE (t=2.05s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.223
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.550
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.145
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.271
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.326
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.326
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.326
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.266
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.507
03/19 21:10:03 - mmengine - INFO - bbox_mAP_copypaste: 0.223 0.550 0.137 0.145 0.271 0.419
03/19 21:10:03 - mmengine - INFO - Epoch(val) [48][250/250]    coco/bbox_mAP: 0.2230  coco/bbox_mAP_50: 0.5500  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1450  coco/bbox_mAP_m: 0.2710  coco/bbox_mAP_l: 0.4190  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:10:15 - mmengine - INFO - Epoch(train)  [49][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:45  time: 0.2321  data_time: 0.0184  memory: 3937  loss: 4.4326  loss_cls: 0.6809  loss_bbox: 2.3605  loss_obj: 1.3912
03/19 21:10:27 - mmengine - INFO - Epoch(train)  [49][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:36  time: 0.2417  data_time: 0.0081  memory: 3937  loss: 4.3717  loss_cls: 0.6793  loss_bbox: 2.3450  loss_obj: 1.3473
03/19 21:10:38 - mmengine - INFO - Epoch(train)  [49][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:26  time: 0.2333  data_time: 0.0082  memory: 3937  loss: 4.4267  loss_cls: 0.6880  loss_bbox: 2.3432  loss_obj: 1.3956
03/19 21:10:49 - mmengine - INFO - Epoch(train)  [49][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:14  time: 0.2034  data_time: 0.0084  memory: 3937  loss: 4.4602  loss_cls: 0.6877  loss_bbox: 2.3914  loss_obj: 1.3811
03/19 21:11:00 - mmengine - INFO - Epoch(train)  [49][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:07:04  time: 0.2353  data_time: 0.0082  memory: 3937  loss: 4.4019  loss_cls: 0.6886  loss_bbox: 2.3443  loss_obj: 1.3691
03/19 21:11:10 - mmengine - INFO - Epoch(train)  [49][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:06:52  time: 0.1977  data_time: 0.0082  memory: 3357  loss: 4.3225  loss_cls: 0.6771  loss_bbox: 2.3403  loss_obj: 1.3052
03/19 21:11:21 - mmengine - INFO - Epoch(train)  [49][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:06:41  time: 0.2135  data_time: 0.0081  memory: 3937  loss: 4.3842  loss_cls: 0.6879  loss_bbox: 2.3636  loss_obj: 1.3327
03/19 21:11:32 - mmengine - INFO - Epoch(train)  [49][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:06:31  time: 0.2235  data_time: 0.0083  memory: 3639  loss: 4.3852  loss_cls: 0.6826  loss_bbox: 2.3517  loss_obj: 1.3509
03/19 21:11:42 - mmengine - INFO - Epoch(train)  [49][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:06:19  time: 0.2007  data_time: 0.0083  memory: 3071  loss: 4.4597  loss_cls: 0.7012  loss_bbox: 2.3950  loss_obj: 1.3635
03/19 21:11:52 - mmengine - INFO - Epoch(train)  [49][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:06:07  time: 0.1932  data_time: 0.0083  memory: 3639  loss: 4.4056  loss_cls: 0.6977  loss_bbox: 2.3721  loss_obj: 1.3358
03/19 21:12:02 - mmengine - INFO - Epoch(train)  [49][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:05:55  time: 0.2026  data_time: 0.0083  memory: 3639  loss: 4.4112  loss_cls: 0.6897  loss_bbox: 2.3661  loss_obj: 1.3555
03/19 21:12:12 - mmengine - INFO - Epoch(train)  [49][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:05:44  time: 0.2050  data_time: 0.0081  memory: 3357  loss: 4.3270  loss_cls: 0.6785  loss_bbox: 2.3270  loss_obj: 1.3214
03/19 21:12:23 - mmengine - INFO - Epoch(train)  [49][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:05:33  time: 0.2133  data_time: 0.0083  memory: 3639  loss: 4.3644  loss_cls: 0.6854  loss_bbox: 2.3418  loss_obj: 1.3372
03/19 21:12:33 - mmengine - INFO - Epoch(train)  [49][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:05:21  time: 0.1963  data_time: 0.0082  memory: 3071  loss: 4.3249  loss_cls: 0.6902  loss_bbox: 2.3378  loss_obj: 1.2969
03/19 21:12:44 - mmengine - INFO - Epoch(train)  [49][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:05:10  time: 0.2159  data_time: 0.0082  memory: 3937  loss: 4.3743  loss_cls: 0.6826  loss_bbox: 2.3564  loss_obj: 1.3354
03/19 21:12:53 - mmengine - INFO - Epoch(train)  [49][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:58  time: 0.1902  data_time: 0.0081  memory: 2338  loss: 4.3372  loss_cls: 0.6736  loss_bbox: 2.3452  loss_obj: 1.3184
03/19 21:13:04 - mmengine - INFO - Epoch(train)  [49][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:47  time: 0.2092  data_time: 0.0082  memory: 3937  loss: 4.3653  loss_cls: 0.6815  loss_bbox: 2.3443  loss_obj: 1.3395
03/19 21:13:15 - mmengine - INFO - Epoch(train)  [49][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:36  time: 0.2219  data_time: 0.0080  memory: 3071  loss: 4.3239  loss_cls: 0.6765  loss_bbox: 2.3363  loss_obj: 1.3111
03/19 21:13:26 - mmengine - INFO - Epoch(train)  [49][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:26  time: 0.2298  data_time: 0.0082  memory: 3937  loss: 4.3628  loss_cls: 0.6832  loss_bbox: 2.3237  loss_obj: 1.3559
03/19 21:13:38 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:13:38 - mmengine - INFO - Epoch(train)  [49][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:17  time: 0.2441  data_time: 0.0080  memory: 3937  loss: 4.4693  loss_cls: 0.6900  loss_bbox: 2.3501  loss_obj: 1.4293
03/19 21:13:38 - mmengine - INFO - Saving checkpoint at 49 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:13:44 - mmengine - INFO - Epoch(val)  [49][ 50/250]    eta: 0:00:11  time: 0.0591  data_time: 0.0072  memory: 527  
03/19 21:13:47 - mmengine - INFO - Epoch(val)  [49][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0066  memory: 527  
03/19 21:13:50 - mmengine - INFO - Epoch(val)  [49][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0064  memory: 527  
03/19 21:13:52 - mmengine - INFO - Epoch(val)  [49][200/250]    eta: 0:00:02  time: 0.0587  data_time: 0.0066  memory: 527  
03/19 21:13:55 - mmengine - INFO - Epoch(val)  [49][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0065  memory: 527  
03/19 21:13:57 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.62s).
Accumulating evaluation results...
DONE (t=2.04s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.224
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.551
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.146
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.270
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.430
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.268
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.378
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.513
03/19 21:14:07 - mmengine - INFO - bbox_mAP_copypaste: 0.224 0.551 0.137 0.146 0.270 0.430
03/19 21:14:07 - mmengine - INFO - Epoch(val) [49][250/250]    coco/bbox_mAP: 0.2240  coco/bbox_mAP_50: 0.5510  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1460  coco/bbox_mAP_m: 0.2700  coco/bbox_mAP_l: 0.4300  data_time: 0.0066  time: 0.0581
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:14:19 - mmengine - INFO - Epoch(train)  [50][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:04:06  time: 0.2282  data_time: 0.0188  memory: 3357  loss: 4.3955  loss_cls: 0.6806  loss_bbox: 2.3322  loss_obj: 1.3827
03/19 21:14:29 - mmengine - INFO - Epoch(train)  [50][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:55  time: 0.2152  data_time: 0.0083  memory: 3639  loss: 4.3345  loss_cls: 0.6725  loss_bbox: 2.3404  loss_obj: 1.3216
03/19 21:14:41 - mmengine - INFO - Epoch(train)  [50][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:45  time: 0.2269  data_time: 0.0082  memory: 3937  loss: 4.3668  loss_cls: 0.6805  loss_bbox: 2.3391  loss_obj: 1.3472
03/19 21:14:51 - mmengine - INFO - Epoch(train)  [50][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:34  time: 0.2123  data_time: 0.0081  memory: 3639  loss: 4.3861  loss_cls: 0.6861  loss_bbox: 2.3542  loss_obj: 1.3458
03/19 21:15:02 - mmengine - INFO - Epoch(train)  [50][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:23  time: 0.2103  data_time: 0.0082  memory: 3071  loss: 4.3866  loss_cls: 0.6805  loss_bbox: 2.3599  loss_obj: 1.3462
03/19 21:15:12 - mmengine - INFO - Epoch(train)  [50][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:11  time: 0.1965  data_time: 0.0084  memory: 3357  loss: 4.3586  loss_cls: 0.6869  loss_bbox: 2.3634  loss_obj: 1.3083
03/19 21:15:24 - mmengine - INFO - Epoch(train)  [50][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:03:02  time: 0.2492  data_time: 0.0081  memory: 3937  loss: 4.4481  loss_cls: 0.6869  loss_bbox: 2.3533  loss_obj: 1.4079
03/19 21:15:35 - mmengine - INFO - Epoch(train)  [50][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:02:51  time: 0.2101  data_time: 0.0083  memory: 3357  loss: 4.4544  loss_cls: 0.6963  loss_bbox: 2.3661  loss_obj: 1.3921
03/19 21:15:46 - mmengine - INFO - Epoch(train)  [50][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:02:40  time: 0.2314  data_time: 0.0082  memory: 3937  loss: 4.3261  loss_cls: 0.6704  loss_bbox: 2.3253  loss_obj: 1.3304
03/19 21:15:57 - mmengine - INFO - Epoch(train)  [50][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:02:29  time: 0.2125  data_time: 0.0081  memory: 3937  loss: 4.3703  loss_cls: 0.6815  loss_bbox: 2.3563  loss_obj: 1.3326
03/19 21:16:08 - mmengine - INFO - Epoch(train)  [50][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:02:19  time: 0.2228  data_time: 0.0081  memory: 3937  loss: 4.3868  loss_cls: 0.6884  loss_bbox: 2.3399  loss_obj: 1.3585
03/19 21:16:19 - mmengine - INFO - Epoch(train)  [50][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:02:08  time: 0.2140  data_time: 0.0083  memory: 3639  loss: 4.3540  loss_cls: 0.6777  loss_bbox: 2.3450  loss_obj: 1.3313
03/19 21:16:28 - mmengine - INFO - Epoch(train)  [50][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:56  time: 0.1946  data_time: 0.0084  memory: 3357  loss: 4.4163  loss_cls: 0.6893  loss_bbox: 2.3731  loss_obj: 1.3539
03/19 21:16:40 - mmengine - INFO - Epoch(train)  [50][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:45  time: 0.2220  data_time: 0.0082  memory: 3357  loss: 4.4121  loss_cls: 0.6787  loss_bbox: 2.3424  loss_obj: 1.3911
03/19 21:16:50 - mmengine - INFO - Epoch(train)  [50][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:34  time: 0.2148  data_time: 0.0083  memory: 3639  loss: 4.4353  loss_cls: 0.6908  loss_bbox: 2.3633  loss_obj: 1.3812
03/19 21:17:02 - mmengine - INFO - Epoch(train)  [50][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:24  time: 0.2279  data_time: 0.0082  memory: 3639  loss: 4.3301  loss_cls: 0.6630  loss_bbox: 2.3177  loss_obj: 1.3494
03/19 21:17:12 - mmengine - INFO - Epoch(train)  [50][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:13  time: 0.2139  data_time: 0.0082  memory: 3937  loss: 4.3287  loss_cls: 0.6819  loss_bbox: 2.3282  loss_obj: 1.3186
03/19 21:17:23 - mmengine - INFO - Epoch(train)  [50][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:01:02  time: 0.2204  data_time: 0.0082  memory: 3937  loss: 4.4319  loss_cls: 0.6892  loss_bbox: 2.3698  loss_obj: 1.3729
03/19 21:17:34 - mmengine - INFO - Epoch(train)  [50][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:00:51  time: 0.2111  data_time: 0.0082  memory: 3639  loss: 4.3971  loss_cls: 0.6867  loss_bbox: 2.3466  loss_obj: 1.3638
03/19 21:17:45 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:17:45 - mmengine - INFO - Epoch(train)  [50][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:00:40  time: 0.2119  data_time: 0.0083  memory: 3937  loss: 4.3511  loss_cls: 0.6767  loss_bbox: 2.3412  loss_obj: 1.3332
03/19 21:17:45 - mmengine - INFO - Saving checkpoint at 50 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:17:50 - mmengine - INFO - Epoch(val)  [50][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 21:17:53 - mmengine - INFO - Epoch(val)  [50][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 21:17:56 - mmengine - INFO - Epoch(val)  [50][150/250]    eta: 0:00:05  time: 0.0578  data_time: 0.0064  memory: 527  
03/19 21:17:59 - mmengine - INFO - Epoch(val)  [50][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0066  memory: 527  
03/19 21:18:02 - mmengine - INFO - Epoch(val)  [50][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0066  memory: 527  
03/19 21:18:03 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.43s).
Accumulating evaluation results...
DONE (t=2.03s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.225
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.551
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.137
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.147
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.271
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.449
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.268
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.515
03/19 21:18:13 - mmengine - INFO - bbox_mAP_copypaste: 0.225 0.551 0.137 0.147 0.271 0.449
03/19 21:18:13 - mmengine - INFO - Epoch(val) [50][250/250]    coco/bbox_mAP: 0.2250  coco/bbox_mAP_50: 0.5510  coco/bbox_mAP_75: 0.1370  coco/bbox_mAP_s: 0.1470  coco/bbox_mAP_m: 0.2710  coco/bbox_mAP_l: 0.4490  data_time: 0.0067  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:18:25 - mmengine - INFO - Epoch(train)  [51][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:00:31  time: 0.2403  data_time: 0.0191  memory: 3937  loss: 4.3998  loss_cls: 0.6808  loss_bbox: 2.3467  loss_obj: 1.3723
03/19 21:18:36 - mmengine - INFO - Epoch(train)  [51][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:00:19  time: 0.2038  data_time: 0.0083  memory: 3357  loss: 4.3465  loss_cls: 0.6790  loss_bbox: 2.3184  loss_obj: 1.3491
03/19 21:18:46 - mmengine - INFO - Epoch(train)  [51][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 3:00:08  time: 0.2034  data_time: 0.0082  memory: 2817  loss: 4.3783  loss_cls: 0.6905  loss_bbox: 2.3479  loss_obj: 1.3398
03/19 21:18:57 - mmengine - INFO - Epoch(train)  [51][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:57  time: 0.2161  data_time: 0.0083  memory: 3639  loss: 4.3523  loss_cls: 0.6716  loss_bbox: 2.3331  loss_obj: 1.3476
03/19 21:19:07 - mmengine - INFO - Epoch(train)  [51][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:46  time: 0.2118  data_time: 0.0082  memory: 3937  loss: 4.3611  loss_cls: 0.6889  loss_bbox: 2.3562  loss_obj: 1.3159
03/19 21:19:18 - mmengine - INFO - Epoch(train)  [51][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:35  time: 0.2221  data_time: 0.0083  memory: 3937  loss: 4.3322  loss_cls: 0.6801  loss_bbox: 2.3278  loss_obj: 1.3243
03/19 21:19:28 - mmengine - INFO - Epoch(train)  [51][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:23  time: 0.1936  data_time: 0.0084  memory: 3357  loss: 4.4332  loss_cls: 0.7006  loss_bbox: 2.3713  loss_obj: 1.3613
03/19 21:19:38 - mmengine - INFO - Epoch(train)  [51][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:11  time: 0.2017  data_time: 0.0081  memory: 3071  loss: 4.4193  loss_cls: 0.6900  loss_bbox: 2.3694  loss_obj: 1.3599
03/19 21:19:48 - mmengine - INFO - Epoch(train)  [51][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:59:00  time: 0.2074  data_time: 0.0082  memory: 3937  loss: 4.3594  loss_cls: 0.6872  loss_bbox: 2.3652  loss_obj: 1.3071
03/19 21:19:59 - mmengine - INFO - Epoch(train)  [51][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:58:49  time: 0.2054  data_time: 0.0082  memory: 3071  loss: 4.3510  loss_cls: 0.6800  loss_bbox: 2.3424  loss_obj: 1.3286
03/19 21:20:10 - mmengine - INFO - Epoch(train)  [51][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:58:39  time: 0.2338  data_time: 0.0082  memory: 3937  loss: 4.3337  loss_cls: 0.6742  loss_bbox: 2.3247  loss_obj: 1.3348
03/19 21:20:22 - mmengine - INFO - Epoch(train)  [51][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:58:29  time: 0.2337  data_time: 0.0084  memory: 3937  loss: 4.3918  loss_cls: 0.6826  loss_bbox: 2.3435  loss_obj: 1.3657
03/19 21:20:32 - mmengine - INFO - Epoch(train)  [51][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:58:17  time: 0.2040  data_time: 0.0084  memory: 3937  loss: 4.3751  loss_cls: 0.6896  loss_bbox: 2.3504  loss_obj: 1.3351
03/19 21:20:43 - mmengine - INFO - Epoch(train)  [51][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:58:06  time: 0.2174  data_time: 0.0083  memory: 3357  loss: 4.3914  loss_cls: 0.6894  loss_bbox: 2.3480  loss_obj: 1.3540
03/19 21:20:54 - mmengine - INFO - Epoch(train)  [51][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:56  time: 0.2228  data_time: 0.0082  memory: 3937  loss: 4.4191  loss_cls: 0.6856  loss_bbox: 2.3563  loss_obj: 1.3772
03/19 21:21:06 - mmengine - INFO - Epoch(train)  [51][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:46  time: 0.2270  data_time: 0.0081  memory: 3937  loss: 4.3309  loss_cls: 0.6780  loss_bbox: 2.3576  loss_obj: 1.2954
03/19 21:21:17 - mmengine - INFO - Epoch(train)  [51][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:35  time: 0.2265  data_time: 0.0082  memory: 3937  loss: 4.4004  loss_cls: 0.6815  loss_bbox: 2.3391  loss_obj: 1.3798
03/19 21:21:29 - mmengine - INFO - Epoch(train)  [51][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:25  time: 0.2310  data_time: 0.0083  memory: 3937  loss: 4.3300  loss_cls: 0.6699  loss_bbox: 2.3345  loss_obj: 1.3257
03/19 21:21:42 - mmengine - INFO - Epoch(train)  [51][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:16  time: 0.2618  data_time: 0.0082  memory: 3937  loss: 4.3182  loss_cls: 0.6668  loss_bbox: 2.3128  loss_obj: 1.3385
03/19 21:21:54 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:21:54 - mmengine - INFO - Epoch(train)  [51][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:57:06  time: 0.2359  data_time: 0.0080  memory: 3937  loss: 4.2971  loss_cls: 0.6691  loss_bbox: 2.3092  loss_obj: 1.3188
03/19 21:21:54 - mmengine - INFO - Saving checkpoint at 51 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:21:59 - mmengine - INFO - Epoch(val)  [51][ 50/250]    eta: 0:00:11  time: 0.0598  data_time: 0.0073  memory: 527  
03/19 21:22:02 - mmengine - INFO - Epoch(val)  [51][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 21:22:05 - mmengine - INFO - Epoch(val)  [51][150/250]    eta: 0:00:05  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 21:22:08 - mmengine - INFO - Epoch(val)  [51][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0066  memory: 527  
03/19 21:22:11 - mmengine - INFO - Epoch(val)  [51][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0065  memory: 527  
03/19 21:22:12 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.10s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.54s).
Accumulating evaluation results...
DONE (t=2.02s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.226
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.550
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.139
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.148
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.269
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.440
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.267
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.379
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.521
03/19 21:22:22 - mmengine - INFO - bbox_mAP_copypaste: 0.226 0.550 0.139 0.148 0.269 0.440
03/19 21:22:22 - mmengine - INFO - Epoch(val) [51][250/250]    coco/bbox_mAP: 0.2260  coco/bbox_mAP_50: 0.5500  coco/bbox_mAP_75: 0.1390  coco/bbox_mAP_s: 0.1480  coco/bbox_mAP_m: 0.2690  coco/bbox_mAP_l: 0.4400  data_time: 0.0067  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:22:34 - mmengine - INFO - Epoch(train)  [52][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:57  time: 0.2452  data_time: 0.0187  memory: 3639  loss: 4.3564  loss_cls: 0.6770  loss_bbox: 2.3329  loss_obj: 1.3465
03/19 21:22:45 - mmengine - INFO - Epoch(train)  [52][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:46  time: 0.2165  data_time: 0.0081  memory: 3071  loss: 4.3604  loss_cls: 0.6746  loss_bbox: 2.3493  loss_obj: 1.3366
03/19 21:22:55 - mmengine - INFO - Epoch(train)  [52][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:34  time: 0.1954  data_time: 0.0085  memory: 3357  loss: 4.3665  loss_cls: 0.6964  loss_bbox: 2.3510  loss_obj: 1.3191
03/19 21:23:07 - mmengine - INFO - Epoch(train)  [52][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:24  time: 0.2345  data_time: 0.0082  memory: 3937  loss: 4.4305  loss_cls: 0.6812  loss_bbox: 2.3539  loss_obj: 1.3954
03/19 21:23:19 - mmengine - INFO - Epoch(train)  [52][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:15  time: 0.2413  data_time: 0.0082  memory: 3937  loss: 4.3779  loss_cls: 0.6774  loss_bbox: 2.3457  loss_obj: 1.3549
03/19 21:23:29 - mmengine - INFO - Epoch(train)  [52][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:56:03  time: 0.2053  data_time: 0.0082  memory: 3357  loss: 4.4297  loss_cls: 0.6888  loss_bbox: 2.3750  loss_obj: 1.3659
03/19 21:23:40 - mmengine - INFO - Epoch(train)  [52][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:55:52  time: 0.2143  data_time: 0.0085  memory: 3937  loss: 4.2865  loss_cls: 0.6668  loss_bbox: 2.3305  loss_obj: 1.2892
03/19 21:23:51 - mmengine - INFO - Epoch(train)  [52][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:55:41  time: 0.2182  data_time: 0.0082  memory: 3071  loss: 4.3902  loss_cls: 0.6857  loss_bbox: 2.3668  loss_obj: 1.3377
03/19 21:24:01 - mmengine - INFO - Epoch(train)  [52][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:55:30  time: 0.2091  data_time: 0.0082  memory: 3357  loss: 4.3725  loss_cls: 0.6821  loss_bbox: 2.3749  loss_obj: 1.3155
03/19 21:24:12 - mmengine - INFO - Epoch(train)  [52][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:55:19  time: 0.2166  data_time: 0.0083  memory: 3937  loss: 4.3695  loss_cls: 0.6816  loss_bbox: 2.3504  loss_obj: 1.3374
03/19 21:24:23 - mmengine - INFO - Epoch(train)  [52][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:55:08  time: 0.2145  data_time: 0.0084  memory: 3639  loss: 4.4193  loss_cls: 0.6870  loss_bbox: 2.3693  loss_obj: 1.3629
03/19 21:24:33 - mmengine - INFO - Epoch(train)  [52][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:57  time: 0.2045  data_time: 0.0082  memory: 3639  loss: 4.3759  loss_cls: 0.6874  loss_bbox: 2.3418  loss_obj: 1.3467
03/19 21:24:44 - mmengine - INFO - Epoch(train)  [52][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:46  time: 0.2095  data_time: 0.0083  memory: 3937  loss: 4.4258  loss_cls: 0.6962  loss_bbox: 2.3684  loss_obj: 1.3612
03/19 21:24:54 - mmengine - INFO - Epoch(train)  [52][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:35  time: 0.2120  data_time: 0.0083  memory: 3357  loss: 4.3351  loss_cls: 0.6765  loss_bbox: 2.3395  loss_obj: 1.3190
03/19 21:25:06 - mmengine - INFO - Epoch(train)  [52][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:24  time: 0.2263  data_time: 0.0080  memory: 3937  loss: 4.3490  loss_cls: 0.6821  loss_bbox: 2.3450  loss_obj: 1.3219
03/19 21:25:18 - mmengine - INFO - Epoch(train)  [52][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:15  time: 0.2408  data_time: 0.0081  memory: 3937  loss: 4.3866  loss_cls: 0.6722  loss_bbox: 2.3550  loss_obj: 1.3594
03/19 21:25:29 - mmengine - INFO - Epoch(train)  [52][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:54:04  time: 0.2223  data_time: 0.0082  memory: 3639  loss: 4.3823  loss_cls: 0.6843  loss_bbox: 2.3490  loss_obj: 1.3490
03/19 21:25:40 - mmengine - INFO - Epoch(train)  [52][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:53:53  time: 0.2228  data_time: 0.0082  memory: 3937  loss: 4.3962  loss_cls: 0.6823  loss_bbox: 2.3412  loss_obj: 1.3727
03/19 21:25:50 - mmengine - INFO - Epoch(train)  [52][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:53:42  time: 0.1981  data_time: 0.0083  memory: 3071  loss: 4.3743  loss_cls: 0.6918  loss_bbox: 2.3663  loss_obj: 1.3163
03/19 21:26:00 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:26:00 - mmengine - INFO - Epoch(train)  [52][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:53:30  time: 0.2062  data_time: 0.0082  memory: 3937  loss: 4.3562  loss_cls: 0.6815  loss_bbox: 2.3409  loss_obj: 1.3339
03/19 21:26:00 - mmengine - INFO - Saving checkpoint at 52 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:26:05 - mmengine - INFO - Epoch(val)  [52][ 50/250]    eta: 0:00:11  time: 0.0587  data_time: 0.0071  memory: 527  
03/19 21:26:08 - mmengine - INFO - Epoch(val)  [52][100/250]    eta: 0:00:08  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 21:26:11 - mmengine - INFO - Epoch(val)  [52][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 21:26:14 - mmengine - INFO - Epoch(val)  [52][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0066  memory: 527  
03/19 21:26:17 - mmengine - INFO - Epoch(val)  [52][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0065  memory: 527  
03/19 21:26:18 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.27s).
Accumulating evaluation results...
DONE (t=1.99s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.552
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.141
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.269
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.435
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.269
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.519
03/19 21:26:28 - mmengine - INFO - bbox_mAP_copypaste: 0.227 0.552 0.141 0.149 0.269 0.435
03/19 21:26:29 - mmengine - INFO - Epoch(val) [52][250/250]    coco/bbox_mAP: 0.2270  coco/bbox_mAP_50: 0.5520  coco/bbox_mAP_75: 0.1410  coco/bbox_mAP_s: 0.1490  coco/bbox_mAP_m: 0.2690  coco/bbox_mAP_l: 0.4350  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:26:40 - mmengine - INFO - Epoch(train)  [53][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:53:20  time: 0.2307  data_time: 0.0190  memory: 3071  loss: 4.4261  loss_cls: 0.6877  loss_bbox: 2.3752  loss_obj: 1.3632
03/19 21:26:51 - mmengine - INFO - Epoch(train)  [53][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:53:10  time: 0.2251  data_time: 0.0082  memory: 3357  loss: 4.3175  loss_cls: 0.6703  loss_bbox: 2.3330  loss_obj: 1.3142
03/19 21:27:02 - mmengine - INFO - Epoch(train)  [53][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:58  time: 0.2088  data_time: 0.0083  memory: 3937  loss: 4.3445  loss_cls: 0.6817  loss_bbox: 2.3219  loss_obj: 1.3409
03/19 21:27:13 - mmengine - INFO - Epoch(train)  [53][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:48  time: 0.2304  data_time: 0.0082  memory: 3937  loss: 4.4297  loss_cls: 0.6897  loss_bbox: 2.3463  loss_obj: 1.3937
03/19 21:27:25 - mmengine - INFO - Epoch(train)  [53][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:38  time: 0.2254  data_time: 0.0081  memory: 3639  loss: 4.3739  loss_cls: 0.6729  loss_bbox: 2.3427  loss_obj: 1.3584
03/19 21:27:36 - mmengine - INFO - Epoch(train)  [53][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:27  time: 0.2269  data_time: 0.0080  memory: 3639  loss: 4.3030  loss_cls: 0.6717  loss_bbox: 2.3096  loss_obj: 1.3216
03/19 21:27:47 - mmengine - INFO - Epoch(train)  [53][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:16  time: 0.2095  data_time: 0.0080  memory: 3071  loss: 4.3641  loss_cls: 0.6759  loss_bbox: 2.3683  loss_obj: 1.3200
03/19 21:27:58 - mmengine - INFO - Epoch(train)  [53][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:52:06  time: 0.2272  data_time: 0.0082  memory: 3357  loss: 4.3860  loss_cls: 0.6804  loss_bbox: 2.3251  loss_obj: 1.3806
03/19 21:28:09 - mmengine - INFO - Epoch(train)  [53][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:55  time: 0.2169  data_time: 0.0083  memory: 3639  loss: 4.3255  loss_cls: 0.6821  loss_bbox: 2.3215  loss_obj: 1.3219
03/19 21:28:20 - mmengine - INFO - Epoch(train)  [53][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:44  time: 0.2186  data_time: 0.0081  memory: 3639  loss: 4.3512  loss_cls: 0.6725  loss_bbox: 2.3138  loss_obj: 1.3649
03/19 21:28:30 - mmengine - INFO - Epoch(train)  [53][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:33  time: 0.2022  data_time: 0.0084  memory: 3071  loss: 4.4410  loss_cls: 0.6939  loss_bbox: 2.3775  loss_obj: 1.3695
03/19 21:28:41 - mmengine - INFO - Epoch(train)  [53][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:22  time: 0.2163  data_time: 0.0085  memory: 3937  loss: 4.4312  loss_cls: 0.6943  loss_bbox: 2.3729  loss_obj: 1.3640
03/19 21:28:53 - mmengine - INFO - Epoch(train)  [53][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:12  time: 0.2374  data_time: 0.0082  memory: 3937  loss: 4.3602  loss_cls: 0.6781  loss_bbox: 2.3344  loss_obj: 1.3476
03/19 21:29:03 - mmengine - INFO - Epoch(train)  [53][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:51:00  time: 0.1998  data_time: 0.0085  memory: 3937  loss: 4.3266  loss_cls: 0.6740  loss_bbox: 2.3389  loss_obj: 1.3137
03/19 21:29:14 - mmengine - INFO - Epoch(train)  [53][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:50:50  time: 0.2271  data_time: 0.0081  memory: 3937  loss: 4.3937  loss_cls: 0.6873  loss_bbox: 2.3486  loss_obj: 1.3579
03/19 21:29:24 - mmengine - INFO - Epoch(train)  [53][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:50:39  time: 0.2070  data_time: 0.0082  memory: 3639  loss: 4.4327  loss_cls: 0.6933  loss_bbox: 2.3784  loss_obj: 1.3611
03/19 21:29:36 - mmengine - INFO - Epoch(train)  [53][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:50:29  time: 0.2356  data_time: 0.0083  memory: 3639  loss: 4.3890  loss_cls: 0.6726  loss_bbox: 2.3470  loss_obj: 1.3694
03/19 21:29:47 - mmengine - INFO - Epoch(train)  [53][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:50:18  time: 0.2254  data_time: 0.0082  memory: 3639  loss: 4.3298  loss_cls: 0.6751  loss_bbox: 2.3200  loss_obj: 1.3346
03/19 21:29:57 - mmengine - INFO - Epoch(train)  [53][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:50:06  time: 0.1973  data_time: 0.0081  memory: 2587  loss: 4.3456  loss_cls: 0.6834  loss_bbox: 2.3476  loss_obj: 1.3147
03/19 21:30:08 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:30:08 - mmengine - INFO - Epoch(train)  [53][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:55  time: 0.2132  data_time: 0.0081  memory: 3639  loss: 4.3499  loss_cls: 0.6809  loss_bbox: 2.3470  loss_obj: 1.3220
03/19 21:30:08 - mmengine - INFO - Saving checkpoint at 53 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:30:13 - mmengine - INFO - Epoch(val)  [53][ 50/250]    eta: 0:00:11  time: 0.0588  data_time: 0.0071  memory: 527  
03/19 21:30:16 - mmengine - INFO - Epoch(val)  [53][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 21:30:19 - mmengine - INFO - Epoch(val)  [53][150/250]    eta: 0:00:05  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 21:30:22 - mmengine - INFO - Epoch(val)  [53][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 21:30:25 - mmengine - INFO - Epoch(val)  [53][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0064  memory: 527  
03/19 21:30:26 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.09s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.46s).
Accumulating evaluation results...
DONE (t=2.00s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.555
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.140
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.273
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.451
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.327
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.327
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.327
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.266
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.395
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.515
03/19 21:30:36 - mmengine - INFO - bbox_mAP_copypaste: 0.227 0.555 0.140 0.149 0.273 0.451
03/19 21:30:36 - mmengine - INFO - Epoch(val) [53][250/250]    coco/bbox_mAP: 0.2270  coco/bbox_mAP_50: 0.5550  coco/bbox_mAP_75: 0.1400  coco/bbox_mAP_s: 0.1490  coco/bbox_mAP_m: 0.2730  coco/bbox_mAP_l: 0.4510  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:30:47 - mmengine - INFO - Epoch(train)  [54][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:44  time: 0.2154  data_time: 0.0183  memory: 3357  loss: 4.3384  loss_cls: 0.6823  loss_bbox: 2.3364  loss_obj: 1.3197
03/19 21:30:58 - mmengine - INFO - Epoch(train)  [54][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:33  time: 0.2084  data_time: 0.0083  memory: 3357  loss: 4.4013  loss_cls: 0.6874  loss_bbox: 2.3744  loss_obj: 1.3395
03/19 21:31:09 - mmengine - INFO - Epoch(train)  [54][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:23  time: 0.2244  data_time: 0.0081  memory: 3639  loss: 4.3897  loss_cls: 0.6778  loss_bbox: 2.3417  loss_obj: 1.3702
03/19 21:31:19 - mmengine - INFO - Epoch(train)  [54][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:11  time: 0.1981  data_time: 0.0082  memory: 3639  loss: 4.3329  loss_cls: 0.6838  loss_bbox: 2.3397  loss_obj: 1.3094
03/19 21:31:31 - mmengine - INFO - Epoch(train)  [54][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:49:01  time: 0.2358  data_time: 0.0081  memory: 3937  loss: 4.4344  loss_cls: 0.6839  loss_bbox: 2.3585  loss_obj: 1.3920
03/19 21:31:41 - mmengine - INFO - Epoch(train)  [54][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:48:50  time: 0.2119  data_time: 0.0081  memory: 3071  loss: 4.3299  loss_cls: 0.6778  loss_bbox: 2.3470  loss_obj: 1.3051
03/19 21:31:52 - mmengine - INFO - Epoch(train)  [54][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:48:39  time: 0.2063  data_time: 0.0082  memory: 3937  loss: 4.3358  loss_cls: 0.6854  loss_bbox: 2.3631  loss_obj: 1.2873
03/19 21:32:02 - mmengine - INFO - Epoch(train)  [54][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:48:27  time: 0.2051  data_time: 0.0081  memory: 2587  loss: 4.4046  loss_cls: 0.6863  loss_bbox: 2.3561  loss_obj: 1.3622
03/19 21:32:12 - mmengine - INFO - Epoch(train)  [54][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:48:16  time: 0.2060  data_time: 0.0082  memory: 3937  loss: 4.3983  loss_cls: 0.6881  loss_bbox: 2.3771  loss_obj: 1.3330
03/19 21:32:22 - mmengine - INFO - Epoch(train)  [54][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:48:04  time: 0.1881  data_time: 0.0082  memory: 2337  loss: 4.4011  loss_cls: 0.6881  loss_bbox: 2.3785  loss_obj: 1.3346
03/19 21:32:32 - mmengine - INFO - Epoch(train)  [54][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:47:53  time: 0.2105  data_time: 0.0081  memory: 3071  loss: 4.3934  loss_cls: 0.6833  loss_bbox: 2.3590  loss_obj: 1.3511
03/19 21:32:45 - mmengine - INFO - Epoch(train)  [54][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:47:43  time: 0.2504  data_time: 0.0081  memory: 3937  loss: 4.3713  loss_cls: 0.6799  loss_bbox: 2.3234  loss_obj: 1.3680
03/19 21:32:55 - mmengine - INFO - Epoch(train)  [54][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:47:32  time: 0.1971  data_time: 0.0082  memory: 2587  loss: 4.3673  loss_cls: 0.6853  loss_bbox: 2.3448  loss_obj: 1.3373
03/19 21:33:04 - mmengine - INFO - Epoch(train)  [54][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:47:20  time: 0.1962  data_time: 0.0082  memory: 2587  loss: 4.2942  loss_cls: 0.6747  loss_bbox: 2.3516  loss_obj: 1.2679
03/19 21:33:15 - mmengine - INFO - Epoch(train)  [54][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:47:09  time: 0.2065  data_time: 0.0081  memory: 3937  loss: 4.4059  loss_cls: 0.6961  loss_bbox: 2.3738  loss_obj: 1.3361
03/19 21:33:25 - mmengine - INFO - Epoch(train)  [54][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:57  time: 0.1989  data_time: 0.0083  memory: 3071  loss: 4.3365  loss_cls: 0.6813  loss_bbox: 2.3548  loss_obj: 1.3004
03/19 21:33:36 - mmengine - INFO - Epoch(train)  [54][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:46  time: 0.2170  data_time: 0.0081  memory: 3937  loss: 4.5085  loss_cls: 0.6942  loss_bbox: 2.4001  loss_obj: 1.4142
03/19 21:33:47 - mmengine - INFO - Epoch(train)  [54][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:36  time: 0.2324  data_time: 0.0082  memory: 3639  loss: 4.4013  loss_cls: 0.6814  loss_bbox: 2.3628  loss_obj: 1.3571
03/19 21:33:58 - mmengine - INFO - Epoch(train)  [54][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:25  time: 0.2090  data_time: 0.0082  memory: 3357  loss: 4.3928  loss_cls: 0.6884  loss_bbox: 2.3587  loss_obj: 1.3457
03/19 21:34:09 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:34:09 - mmengine - INFO - Epoch(train)  [54][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:14  time: 0.2259  data_time: 0.0081  memory: 3357  loss: 4.3222  loss_cls: 0.6737  loss_bbox: 2.3278  loss_obj: 1.3207
03/19 21:34:09 - mmengine - INFO - Saving checkpoint at 54 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:34:14 - mmengine - INFO - Epoch(val)  [54][ 50/250]    eta: 0:00:12  time: 0.0600  data_time: 0.0072  memory: 527  
03/19 21:34:17 - mmengine - INFO - Epoch(val)  [54][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 21:34:20 - mmengine - INFO - Epoch(val)  [54][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0064  memory: 527  
03/19 21:34:23 - mmengine - INFO - Epoch(val)  [54][200/250]    eta: 0:00:02  time: 0.0592  data_time: 0.0064  memory: 527  
03/19 21:34:26 - mmengine - INFO - Epoch(val)  [54][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0065  memory: 527  
03/19 21:34:27 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.49s).
Accumulating evaluation results...
DONE (t=1.99s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.228
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.555
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.141
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.275
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.451
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.328
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.265
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.402
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.518
03/19 21:34:37 - mmengine - INFO - bbox_mAP_copypaste: 0.228 0.555 0.141 0.149 0.275 0.451
03/19 21:34:37 - mmengine - INFO - Epoch(val) [54][250/250]    coco/bbox_mAP: 0.2280  coco/bbox_mAP_50: 0.5550  coco/bbox_mAP_75: 0.1410  coco/bbox_mAP_s: 0.1490  coco/bbox_mAP_m: 0.2750  coco/bbox_mAP_l: 0.4510  data_time: 0.0066  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:34:49 - mmengine - INFO - Epoch(train)  [55][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:46:04  time: 0.2386  data_time: 0.0189  memory: 3639  loss: 4.2442  loss_cls: 0.6668  loss_bbox: 2.3019  loss_obj: 1.2755
03/19 21:35:01 - mmengine - INFO - Epoch(train)  [55][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:45:54  time: 0.2295  data_time: 0.0081  memory: 3937  loss: 4.3113  loss_cls: 0.6709  loss_bbox: 2.3288  loss_obj: 1.3115
03/19 21:35:11 - mmengine - INFO - Epoch(train)  [55][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:45:43  time: 0.2088  data_time: 0.0080  memory: 3639  loss: 4.3597  loss_cls: 0.6765  loss_bbox: 2.3329  loss_obj: 1.3502
03/19 21:35:21 - mmengine - INFO - Epoch(train)  [55][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:45:31  time: 0.1913  data_time: 0.0084  memory: 2587  loss: 4.3005  loss_cls: 0.6797  loss_bbox: 2.3143  loss_obj: 1.3065
03/19 21:35:32 - mmengine - INFO - Epoch(train)  [55][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:45:20  time: 0.2232  data_time: 0.0083  memory: 3937  loss: 4.4070  loss_cls: 0.6843  loss_bbox: 2.3584  loss_obj: 1.3642
03/19 21:35:42 - mmengine - INFO - Epoch(train)  [55][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:45:09  time: 0.2027  data_time: 0.0083  memory: 3071  loss: 4.2960  loss_cls: 0.6772  loss_bbox: 2.3220  loss_obj: 1.2968
03/19 21:35:52 - mmengine - INFO - Epoch(train)  [55][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:57  time: 0.1974  data_time: 0.0083  memory: 2587  loss: 4.4160  loss_cls: 0.6937  loss_bbox: 2.3713  loss_obj: 1.3509
03/19 21:36:03 - mmengine - INFO - Epoch(train)  [55][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:46  time: 0.2163  data_time: 0.0081  memory: 3639  loss: 4.3863  loss_cls: 0.6821  loss_bbox: 2.3677  loss_obj: 1.3364
03/19 21:36:14 - mmengine - INFO - Epoch(train)  [55][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:36  time: 0.2241  data_time: 0.0081  memory: 3937  loss: 4.4183  loss_cls: 0.6867  loss_bbox: 2.3560  loss_obj: 1.3756
03/19 21:36:25 - mmengine - INFO - Epoch(train)  [55][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:25  time: 0.2082  data_time: 0.0082  memory: 3639  loss: 4.3896  loss_cls: 0.6923  loss_bbox: 2.3546  loss_obj: 1.3427
03/19 21:36:36 - mmengine - INFO - Epoch(train)  [55][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:15  time: 0.2364  data_time: 0.0082  memory: 3639  loss: 4.3485  loss_cls: 0.6750  loss_bbox: 2.3391  loss_obj: 1.3344
03/19 21:36:46 - mmengine - INFO - Epoch(train)  [55][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:44:03  time: 0.1953  data_time: 0.0084  memory: 3357  loss: 4.3969  loss_cls: 0.6843  loss_bbox: 2.3782  loss_obj: 1.3343
03/19 21:36:57 - mmengine - INFO - Epoch(train)  [55][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:43:52  time: 0.2166  data_time: 0.0081  memory: 3639  loss: 4.3647  loss_cls: 0.6778  loss_bbox: 2.3395  loss_obj: 1.3473
03/19 21:37:08 - mmengine - INFO - Epoch(train)  [55][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:43:42  time: 0.2218  data_time: 0.0082  memory: 3357  loss: 4.3439  loss_cls: 0.6745  loss_bbox: 2.3229  loss_obj: 1.3465
03/19 21:37:19 - mmengine - INFO - Epoch(train)  [55][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:43:30  time: 0.2103  data_time: 0.0083  memory: 3357  loss: 4.3519  loss_cls: 0.6828  loss_bbox: 2.3380  loss_obj: 1.3312
03/19 21:37:30 - mmengine - INFO - Epoch(train)  [55][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:43:20  time: 0.2204  data_time: 0.0081  memory: 3639  loss: 4.3582  loss_cls: 0.6800  loss_bbox: 2.3390  loss_obj: 1.3391
03/19 21:37:40 - mmengine - INFO - Epoch(train)  [55][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:43:08  time: 0.2063  data_time: 0.0084  memory: 3071  loss: 4.4541  loss_cls: 0.6983  loss_bbox: 2.3741  loss_obj: 1.3817
03/19 21:37:51 - mmengine - INFO - Epoch(train)  [55][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:57  time: 0.2119  data_time: 0.0082  memory: 3639  loss: 4.3475  loss_cls: 0.6912  loss_bbox: 2.3383  loss_obj: 1.3180
03/19 21:38:01 - mmengine - INFO - Epoch(train)  [55][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:46  time: 0.2121  data_time: 0.0084  memory: 3357  loss: 4.3234  loss_cls: 0.6747  loss_bbox: 2.3277  loss_obj: 1.3210
03/19 21:38:12 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:38:12 - mmengine - INFO - Epoch(train)  [55][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:35  time: 0.2128  data_time: 0.0081  memory: 3937  loss: 4.3424  loss_cls: 0.6786  loss_bbox: 2.3354  loss_obj: 1.3284
03/19 21:38:12 - mmengine - INFO - Saving checkpoint at 55 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:38:17 - mmengine - INFO - Epoch(val)  [55][ 50/250]    eta: 0:00:11  time: 0.0591  data_time: 0.0072  memory: 527  
03/19 21:38:20 - mmengine - INFO - Epoch(val)  [55][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0066  memory: 527  
03/19 21:38:23 - mmengine - INFO - Epoch(val)  [55][150/250]    eta: 0:00:05  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 21:38:26 - mmengine - INFO - Epoch(val)  [55][200/250]    eta: 0:00:02  time: 0.0594  data_time: 0.0065  memory: 527  
03/19 21:38:29 - mmengine - INFO - Epoch(val)  [55][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 21:38:30 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.44s).
Accumulating evaluation results...
DONE (t=1.99s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.229
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.556
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.142
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.276
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.447
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.267
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.402
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.515
03/19 21:38:40 - mmengine - INFO - bbox_mAP_copypaste: 0.229 0.556 0.142 0.151 0.276 0.447
03/19 21:38:40 - mmengine - INFO - Epoch(val) [55][250/250]    coco/bbox_mAP: 0.2290  coco/bbox_mAP_50: 0.5560  coco/bbox_mAP_75: 0.1420  coco/bbox_mAP_s: 0.1510  coco/bbox_mAP_m: 0.2760  coco/bbox_mAP_l: 0.4470  data_time: 0.0066  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:38:52 - mmengine - INFO - Epoch(train)  [56][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:25  time: 0.2273  data_time: 0.0187  memory: 3937  loss: 4.3685  loss_cls: 0.6830  loss_bbox: 2.3417  loss_obj: 1.3438
03/19 21:39:02 - mmengine - INFO - Epoch(train)  [56][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:14  time: 0.2097  data_time: 0.0083  memory: 3639  loss: 4.2866  loss_cls: 0.6761  loss_bbox: 2.3366  loss_obj: 1.2738
03/19 21:39:14 - mmengine - INFO - Epoch(train)  [56][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:42:03  time: 0.2289  data_time: 0.0081  memory: 3357  loss: 4.3622  loss_cls: 0.6831  loss_bbox: 2.3392  loss_obj: 1.3399
03/19 21:39:24 - mmengine - INFO - Epoch(train)  [56][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:41:52  time: 0.2056  data_time: 0.0082  memory: 3071  loss: 4.3142  loss_cls: 0.6739  loss_bbox: 2.3179  loss_obj: 1.3224
03/19 21:39:36 - mmengine - INFO - Epoch(train)  [56][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:41:42  time: 0.2305  data_time: 0.0081  memory: 3639  loss: 4.3820  loss_cls: 0.6747  loss_bbox: 2.3478  loss_obj: 1.3595
03/19 21:39:46 - mmengine - INFO - Epoch(train)  [56][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:41:31  time: 0.2041  data_time: 0.0083  memory: 3937  loss: 4.3931  loss_cls: 0.6765  loss_bbox: 2.3713  loss_obj: 1.3452
03/19 21:39:56 - mmengine - INFO - Epoch(train)  [56][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:41:19  time: 0.2063  data_time: 0.0083  memory: 3357  loss: 4.4283  loss_cls: 0.6927  loss_bbox: 2.3832  loss_obj: 1.3524
03/19 21:40:07 - mmengine - INFO - Epoch(train)  [56][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:41:09  time: 0.2211  data_time: 0.0082  memory: 3639  loss: 4.3797  loss_cls: 0.6710  loss_bbox: 2.3571  loss_obj: 1.3516
03/19 21:40:18 - mmengine - INFO - Epoch(train)  [56][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:58  time: 0.2138  data_time: 0.0081  memory: 3937  loss: 4.3432  loss_cls: 0.6743  loss_bbox: 2.3540  loss_obj: 1.3149
03/19 21:40:28 - mmengine - INFO - Epoch(train)  [56][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:47  time: 0.2122  data_time: 0.0081  memory: 3071  loss: 4.3544  loss_cls: 0.6838  loss_bbox: 2.3524  loss_obj: 1.3182
03/19 21:40:38 - mmengine - INFO - Epoch(train)  [56][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:35  time: 0.1864  data_time: 0.0084  memory: 2587  loss: 4.3491  loss_cls: 0.6845  loss_bbox: 2.3550  loss_obj: 1.3095
03/19 21:40:49 - mmengine - INFO - Epoch(train)  [56][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:24  time: 0.2241  data_time: 0.0082  memory: 3639  loss: 4.3685  loss_cls: 0.6712  loss_bbox: 2.3382  loss_obj: 1.3591
03/19 21:40:59 - mmengine - INFO - Epoch(train)  [56][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:13  time: 0.2017  data_time: 0.0083  memory: 2817  loss: 4.3995  loss_cls: 0.7004  loss_bbox: 2.3709  loss_obj: 1.3283
03/19 21:41:10 - mmengine - INFO - Epoch(train)  [56][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:40:02  time: 0.2236  data_time: 0.0082  memory: 3937  loss: 4.3608  loss_cls: 0.6789  loss_bbox: 2.3464  loss_obj: 1.3355
03/19 21:41:22 - mmengine - INFO - Epoch(train)  [56][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:39:51  time: 0.2237  data_time: 0.0081  memory: 3937  loss: 4.3584  loss_cls: 0.6742  loss_bbox: 2.3534  loss_obj: 1.3307
03/19 21:41:32 - mmengine - INFO - Epoch(train)  [56][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:39:41  time: 0.2145  data_time: 0.0082  memory: 3357  loss: 4.3336  loss_cls: 0.6764  loss_bbox: 2.3525  loss_obj: 1.3048
03/19 21:41:44 - mmengine - INFO - Epoch(train)  [56][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:39:30  time: 0.2342  data_time: 0.0082  memory: 3937  loss: 4.3691  loss_cls: 0.6828  loss_bbox: 2.3280  loss_obj: 1.3583
03/19 21:41:54 - mmengine - INFO - Epoch(train)  [56][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:39:19  time: 0.1940  data_time: 0.0084  memory: 2817  loss: 4.3586  loss_cls: 0.6873  loss_bbox: 2.3728  loss_obj: 1.2985
03/19 21:42:04 - mmengine - INFO - Epoch(train)  [56][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:39:07  time: 0.2093  data_time: 0.0082  memory: 3071  loss: 4.3436  loss_cls: 0.6804  loss_bbox: 2.3453  loss_obj: 1.3179
03/19 21:42:16 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:42:16 - mmengine - INFO - Epoch(train)  [56][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:57  time: 0.2326  data_time: 0.0081  memory: 3937  loss: 4.3656  loss_cls: 0.6765  loss_bbox: 2.3217  loss_obj: 1.3674
03/19 21:42:16 - mmengine - INFO - Saving checkpoint at 56 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:42:21 - mmengine - INFO - Epoch(val)  [56][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0071  memory: 527  
03/19 21:42:24 - mmengine - INFO - Epoch(val)  [56][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 21:42:27 - mmengine - INFO - Epoch(val)  [56][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 21:42:30 - mmengine - INFO - Epoch(val)  [56][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0064  memory: 527  
03/19 21:42:33 - mmengine - INFO - Epoch(val)  [56][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0065  memory: 527  
03/19 21:42:34 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.47s).
Accumulating evaluation results...
DONE (t=2.00s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.228
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.557
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.143
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.274
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.418
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.266
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.401
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.507
03/19 21:42:44 - mmengine - INFO - bbox_mAP_copypaste: 0.228 0.557 0.143 0.151 0.274 0.418
03/19 21:42:44 - mmengine - INFO - Epoch(val) [56][250/250]    coco/bbox_mAP: 0.2280  coco/bbox_mAP_50: 0.5570  coco/bbox_mAP_75: 0.1430  coco/bbox_mAP_s: 0.1510  coco/bbox_mAP_m: 0.2740  coco/bbox_mAP_l: 0.4180  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:42:56 - mmengine - INFO - Epoch(train)  [57][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:47  time: 0.2403  data_time: 0.0187  memory: 3937  loss: 4.3234  loss_cls: 0.6783  loss_bbox: 2.3118  loss_obj: 1.3333
03/19 21:43:08 - mmengine - INFO - Epoch(train)  [57][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:37  time: 0.2377  data_time: 0.0081  memory: 3937  loss: 4.4116  loss_cls: 0.6791  loss_bbox: 2.3540  loss_obj: 1.3785
03/19 21:43:18 - mmengine - INFO - Epoch(train)  [57][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:26  time: 0.2035  data_time: 0.0083  memory: 3639  loss: 4.4111  loss_cls: 0.6825  loss_bbox: 2.3639  loss_obj: 1.3647
03/19 21:43:29 - mmengine - INFO - Epoch(train)  [57][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:15  time: 0.2058  data_time: 0.0082  memory: 3071  loss: 4.3212  loss_cls: 0.6717  loss_bbox: 2.3539  loss_obj: 1.2956
03/19 21:43:40 - mmengine - INFO - Epoch(train)  [57][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:38:05  time: 0.2332  data_time: 0.0082  memory: 3937  loss: 4.2562  loss_cls: 0.6643  loss_bbox: 2.2804  loss_obj: 1.3115
03/19 21:43:52 - mmengine - INFO - Epoch(train)  [57][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:37:54  time: 0.2211  data_time: 0.0082  memory: 3357  loss: 4.3243  loss_cls: 0.6799  loss_bbox: 2.3208  loss_obj: 1.3236
03/19 21:44:02 - mmengine - INFO - Epoch(train)  [57][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:37:42  time: 0.2001  data_time: 0.0081  memory: 2337  loss: 4.3025  loss_cls: 0.6699  loss_bbox: 2.3230  loss_obj: 1.3096
03/19 21:44:12 - mmengine - INFO - Epoch(train)  [57][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:37:31  time: 0.2050  data_time: 0.0082  memory: 2817  loss: 4.3141  loss_cls: 0.6762  loss_bbox: 2.3139  loss_obj: 1.3241
03/19 21:44:22 - mmengine - INFO - Epoch(train)  [57][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:37:19  time: 0.1953  data_time: 0.0083  memory: 3071  loss: 4.3403  loss_cls: 0.6863  loss_bbox: 2.3429  loss_obj: 1.3111
03/19 21:44:33 - mmengine - INFO - Epoch(train)  [57][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:37:09  time: 0.2224  data_time: 0.0082  memory: 3937  loss: 4.2916  loss_cls: 0.6743  loss_bbox: 2.3228  loss_obj: 1.2945
03/19 21:44:44 - mmengine - INFO - Epoch(train)  [57][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:58  time: 0.2213  data_time: 0.0081  memory: 3357  loss: 4.3219  loss_cls: 0.6779  loss_bbox: 2.3235  loss_obj: 1.3206
03/19 21:44:56 - mmengine - INFO - Epoch(train)  [57][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:48  time: 0.2462  data_time: 0.0081  memory: 3937  loss: 4.3640  loss_cls: 0.6704  loss_bbox: 2.3442  loss_obj: 1.3494
03/19 21:45:07 - mmengine - INFO - Epoch(train)  [57][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:37  time: 0.2132  data_time: 0.0082  memory: 3071  loss: 4.3044  loss_cls: 0.6756  loss_bbox: 2.3216  loss_obj: 1.3072
03/19 21:45:18 - mmengine - INFO - Epoch(train)  [57][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:27  time: 0.2189  data_time: 0.0082  memory: 3639  loss: 4.3409  loss_cls: 0.6720  loss_bbox: 2.3418  loss_obj: 1.3272
03/19 21:45:29 - mmengine - INFO - Epoch(train)  [57][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:16  time: 0.2196  data_time: 0.0081  memory: 3937  loss: 4.3597  loss_cls: 0.6750  loss_bbox: 2.3436  loss_obj: 1.3412
03/19 21:45:40 - mmengine - INFO - Epoch(train)  [57][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:36:05  time: 0.2199  data_time: 0.0082  memory: 3357  loss: 4.3163  loss_cls: 0.6711  loss_bbox: 2.3316  loss_obj: 1.3136
03/19 21:45:50 - mmengine - INFO - Epoch(train)  [57][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:54  time: 0.2042  data_time: 0.0082  memory: 3937  loss: 4.3405  loss_cls: 0.6855  loss_bbox: 2.3532  loss_obj: 1.3018
03/19 21:46:02 - mmengine - INFO - Epoch(train)  [57][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:44  time: 0.2450  data_time: 0.0081  memory: 3937  loss: 4.3321  loss_cls: 0.6703  loss_bbox: 2.3132  loss_obj: 1.3486
03/19 21:46:14 - mmengine - INFO - Epoch(train)  [57][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:34  time: 0.2247  data_time: 0.0082  memory: 3357  loss: 4.3068  loss_cls: 0.6749  loss_bbox: 2.3221  loss_obj: 1.3098
03/19 21:46:23 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:46:23 - mmengine - INFO - Epoch(train)  [57][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:22  time: 0.1980  data_time: 0.0081  memory: 3639  loss: 4.3045  loss_cls: 0.6717  loss_bbox: 2.3367  loss_obj: 1.2962
03/19 21:46:23 - mmengine - INFO - Saving checkpoint at 57 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:46:29 - mmengine - INFO - Epoch(val)  [57][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0071  memory: 527  
03/19 21:46:32 - mmengine - INFO - Epoch(val)  [57][100/250]    eta: 0:00:08  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 21:46:35 - mmengine - INFO - Epoch(val)  [57][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 21:46:38 - mmengine - INFO - Epoch(val)  [57][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 21:46:40 - mmengine - INFO - Epoch(val)  [57][250/250]    eta: 0:00:00  time: 0.0566  data_time: 0.0065  memory: 527  
03/19 21:46:42 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.23s).
Accumulating evaluation results...
DONE (t=2.19s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.230
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.558
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.144
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.154
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.273
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.268
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.396
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.508
03/19 21:46:52 - mmengine - INFO - bbox_mAP_copypaste: 0.230 0.558 0.144 0.154 0.273 0.419
03/19 21:46:52 - mmengine - INFO - Epoch(val) [57][250/250]    coco/bbox_mAP: 0.2300  coco/bbox_mAP_50: 0.5580  coco/bbox_mAP_75: 0.1440  coco/bbox_mAP_s: 0.1540  coco/bbox_mAP_m: 0.2730  coco/bbox_mAP_l: 0.4190  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:47:04 - mmengine - INFO - Epoch(train)  [58][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:12  time: 0.2414  data_time: 0.0190  memory: 3639  loss: 4.3322  loss_cls: 0.6812  loss_bbox: 2.3161  loss_obj: 1.3349
03/19 21:47:14 - mmengine - INFO - Epoch(train)  [58][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:35:01  time: 0.2079  data_time: 0.0083  memory: 3357  loss: 4.3361  loss_cls: 0.6741  loss_bbox: 2.3288  loss_obj: 1.3332
03/19 21:47:24 - mmengine - INFO - Epoch(train)  [58][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:34:49  time: 0.1900  data_time: 0.0082  memory: 2587  loss: 4.3447  loss_cls: 0.6861  loss_bbox: 2.3622  loss_obj: 1.2964
03/19 21:47:34 - mmengine - INFO - Epoch(train)  [58][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:34:38  time: 0.2012  data_time: 0.0083  memory: 3639  loss: 4.3527  loss_cls: 0.6809  loss_bbox: 2.3512  loss_obj: 1.3206
03/19 21:47:44 - mmengine - INFO - Epoch(train)  [58][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:34:27  time: 0.2093  data_time: 0.0084  memory: 3639  loss: 4.3772  loss_cls: 0.6798  loss_bbox: 2.3505  loss_obj: 1.3469
03/19 21:47:56 - mmengine - INFO - Epoch(train)  [58][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:34:16  time: 0.2249  data_time: 0.0083  memory: 3937  loss: 4.4091  loss_cls: 0.6826  loss_bbox: 2.3464  loss_obj: 1.3801
03/19 21:48:06 - mmengine - INFO - Epoch(train)  [58][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:34:05  time: 0.2105  data_time: 0.0083  memory: 3071  loss: 4.4120  loss_cls: 0.6863  loss_bbox: 2.3626  loss_obj: 1.3631
03/19 21:48:17 - mmengine - INFO - Epoch(train)  [58][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:54  time: 0.2081  data_time: 0.0082  memory: 3639  loss: 4.3135  loss_cls: 0.6723  loss_bbox: 2.3334  loss_obj: 1.3078
03/19 21:48:28 - mmengine - INFO - Epoch(train)  [58][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:43  time: 0.2237  data_time: 0.0081  memory: 3639  loss: 4.2980  loss_cls: 0.6705  loss_bbox: 2.3104  loss_obj: 1.3171
03/19 21:48:38 - mmengine - INFO - Epoch(train)  [58][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:32  time: 0.1950  data_time: 0.0083  memory: 3639  loss: 4.4045  loss_cls: 0.6955  loss_bbox: 2.3829  loss_obj: 1.3261
03/19 21:48:49 - mmengine - INFO - Epoch(train)  [58][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:21  time: 0.2167  data_time: 0.0082  memory: 3357  loss: 4.3163  loss_cls: 0.6756  loss_bbox: 2.3341  loss_obj: 1.3066
03/19 21:48:59 - mmengine - INFO - Epoch(train)  [58][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:09  time: 0.2028  data_time: 0.0081  memory: 3071  loss: 4.3690  loss_cls: 0.6871  loss_bbox: 2.3450  loss_obj: 1.3368
03/19 21:49:12 - mmengine - INFO - Epoch(train)  [58][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:33:00  time: 0.2599  data_time: 0.0081  memory: 3937  loss: 4.4130  loss_cls: 0.6802  loss_bbox: 2.3478  loss_obj: 1.3850
03/19 21:49:22 - mmengine - INFO - Epoch(train)  [58][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:32:49  time: 0.2049  data_time: 0.0081  memory: 3357  loss: 4.2753  loss_cls: 0.6745  loss_bbox: 2.3067  loss_obj: 1.2940
03/19 21:49:31 - mmengine - INFO - Epoch(train)  [58][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:32:37  time: 0.1905  data_time: 0.0084  memory: 2817  loss: 4.3553  loss_cls: 0.6825  loss_bbox: 2.3574  loss_obj: 1.3154
03/19 21:49:41 - mmengine - INFO - Epoch(train)  [58][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:32:26  time: 0.1990  data_time: 0.0083  memory: 3071  loss: 4.3487  loss_cls: 0.6801  loss_bbox: 2.3412  loss_obj: 1.3274
03/19 21:49:52 - mmengine - INFO - Epoch(train)  [58][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:32:14  time: 0.2061  data_time: 0.0082  memory: 3639  loss: 4.2871  loss_cls: 0.6733  loss_bbox: 2.3227  loss_obj: 1.2912
03/19 21:50:02 - mmengine - INFO - Epoch(train)  [58][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:32:03  time: 0.2056  data_time: 0.0082  memory: 2817  loss: 4.2992  loss_cls: 0.6744  loss_bbox: 2.3086  loss_obj: 1.3161
03/19 21:50:13 - mmengine - INFO - Epoch(train)  [58][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:31:52  time: 0.2188  data_time: 0.0082  memory: 3937  loss: 4.3057  loss_cls: 0.6702  loss_bbox: 2.3323  loss_obj: 1.3033
03/19 21:50:24 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:50:24 - mmengine - INFO - Epoch(train)  [58][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:31:42  time: 0.2271  data_time: 0.0081  memory: 3937  loss: 4.3686  loss_cls: 0.6776  loss_bbox: 2.3530  loss_obj: 1.3380
03/19 21:50:24 - mmengine - INFO - Saving checkpoint at 58 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:50:30 - mmengine - INFO - Epoch(val)  [58][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0073  memory: 527  
03/19 21:50:32 - mmengine - INFO - Epoch(val)  [58][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 21:50:35 - mmengine - INFO - Epoch(val)  [58][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 21:50:38 - mmengine - INFO - Epoch(val)  [58][200/250]    eta: 0:00:02  time: 0.0620  data_time: 0.0091  memory: 527  
03/19 21:50:41 - mmengine - INFO - Epoch(val)  [58][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0065  memory: 527  
03/19 21:50:43 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.24s).
Accumulating evaluation results...
DONE (t=1.96s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.233
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.558
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.148
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.159
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.274
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.422
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.511
03/19 21:50:52 - mmengine - INFO - bbox_mAP_copypaste: 0.233 0.558 0.148 0.159 0.274 0.422
03/19 21:50:53 - mmengine - INFO - Epoch(val) [58][250/250]    coco/bbox_mAP: 0.2330  coco/bbox_mAP_50: 0.5580  coco/bbox_mAP_75: 0.1480  coco/bbox_mAP_s: 0.1590  coco/bbox_mAP_m: 0.2740  coco/bbox_mAP_l: 0.4220  data_time: 0.0072  time: 0.0590
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:51:03 - mmengine - INFO - Epoch(train)  [59][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:31:31  time: 0.2173  data_time: 0.0187  memory: 2587  loss: 4.2971  loss_cls: 0.6787  loss_bbox: 2.3284  loss_obj: 1.2900
03/19 21:51:14 - mmengine - INFO - Epoch(train)  [59][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:31:20  time: 0.2202  data_time: 0.0081  memory: 3639  loss: 4.4230  loss_cls: 0.6868  loss_bbox: 2.3832  loss_obj: 1.3530
03/19 21:51:26 - mmengine - INFO - Epoch(train)  [59][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:31:10  time: 0.2215  data_time: 0.0081  memory: 3639  loss: 4.3456  loss_cls: 0.6712  loss_bbox: 2.3275  loss_obj: 1.3469
03/19 21:51:36 - mmengine - INFO - Epoch(train)  [59][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:58  time: 0.2021  data_time: 0.0084  memory: 3071  loss: 4.3642  loss_cls: 0.6832  loss_bbox: 2.3604  loss_obj: 1.3206
03/19 21:51:46 - mmengine - INFO - Epoch(train)  [59][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:47  time: 0.2055  data_time: 0.0082  memory: 3071  loss: 4.4003  loss_cls: 0.6842  loss_bbox: 2.3711  loss_obj: 1.3449
03/19 21:51:57 - mmengine - INFO - Epoch(train)  [59][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:36  time: 0.2152  data_time: 0.0082  memory: 3357  loss: 4.2834  loss_cls: 0.6796  loss_bbox: 2.3089  loss_obj: 1.2949
03/19 21:52:09 - mmengine - INFO - Epoch(train)  [59][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:26  time: 0.2391  data_time: 0.0082  memory: 3937  loss: 4.2745  loss_cls: 0.6697  loss_bbox: 2.2919  loss_obj: 1.3129
03/19 21:52:18 - mmengine - INFO - Epoch(train)  [59][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:14  time: 0.1759  data_time: 0.0084  memory: 1907  loss: 4.3727  loss_cls: 0.6883  loss_bbox: 2.3923  loss_obj: 1.2921
03/19 21:52:29 - mmengine - INFO - Epoch(train)  [59][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:30:03  time: 0.2287  data_time: 0.0083  memory: 3937  loss: 4.3595  loss_cls: 0.6794  loss_bbox: 2.3318  loss_obj: 1.3483
03/19 21:52:40 - mmengine - INFO - Epoch(train)  [59][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:29:53  time: 0.2213  data_time: 0.0082  memory: 3357  loss: 4.4058  loss_cls: 0.6808  loss_bbox: 2.3609  loss_obj: 1.3641
03/19 21:52:50 - mmengine - INFO - Epoch(train)  [59][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:29:42  time: 0.2082  data_time: 0.0083  memory: 3937  loss: 4.3988  loss_cls: 0.6851  loss_bbox: 2.3657  loss_obj: 1.3481
03/19 21:53:01 - mmengine - INFO - Epoch(train)  [59][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:29:31  time: 0.2137  data_time: 0.0083  memory: 3639  loss: 4.3148  loss_cls: 0.6764  loss_bbox: 2.3271  loss_obj: 1.3113
03/19 21:53:11 - mmengine - INFO - Epoch(train)  [59][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:29:19  time: 0.1934  data_time: 0.0084  memory: 3071  loss: 4.3602  loss_cls: 0.6777  loss_bbox: 2.3641  loss_obj: 1.3184
03/19 21:53:22 - mmengine - INFO - Epoch(train)  [59][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:29:08  time: 0.2204  data_time: 0.0083  memory: 3937  loss: 4.3906  loss_cls: 0.6879  loss_bbox: 2.3560  loss_obj: 1.3467
03/19 21:53:32 - mmengine - INFO - Epoch(train)  [59][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:57  time: 0.2097  data_time: 0.0081  memory: 3071  loss: 4.3747  loss_cls: 0.6849  loss_bbox: 2.3511  loss_obj: 1.3387
03/19 21:53:44 - mmengine - INFO - Epoch(train)  [59][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:47  time: 0.2272  data_time: 0.0081  memory: 3639  loss: 4.2954  loss_cls: 0.6720  loss_bbox: 2.3164  loss_obj: 1.3070
03/19 21:53:54 - mmengine - INFO - Epoch(train)  [59][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:36  time: 0.2101  data_time: 0.0084  memory: 3357  loss: 4.3907  loss_cls: 0.6873  loss_bbox: 2.3638  loss_obj: 1.3396
03/19 21:54:06 - mmengine - INFO - Epoch(train)  [59][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:26  time: 0.2357  data_time: 0.0083  memory: 3937  loss: 4.2704  loss_cls: 0.6672  loss_bbox: 2.2892  loss_obj: 1.3140
03/19 21:54:16 - mmengine - INFO - Epoch(train)  [59][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:14  time: 0.1921  data_time: 0.0084  memory: 3071  loss: 4.4031  loss_cls: 0.6951  loss_bbox: 2.3736  loss_obj: 1.3344
03/19 21:54:28 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:54:28 - mmengine - INFO - Epoch(train)  [59][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:28:04  time: 0.2384  data_time: 0.0082  memory: 3937  loss: 4.3732  loss_cls: 0.6763  loss_bbox: 2.3320  loss_obj: 1.3648
03/19 21:54:28 - mmengine - INFO - Saving checkpoint at 59 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:54:33 - mmengine - INFO - Epoch(val)  [59][ 50/250]    eta: 0:00:11  time: 0.0600  data_time: 0.0072  memory: 527  
03/19 21:54:36 - mmengine - INFO - Epoch(val)  [59][100/250]    eta: 0:00:08  time: 0.0592  data_time: 0.0065  memory: 527  
03/19 21:54:39 - mmengine - INFO - Epoch(val)  [59][150/250]    eta: 0:00:05  time: 0.0590  data_time: 0.0066  memory: 527  
03/19 21:54:42 - mmengine - INFO - Epoch(val)  [59][200/250]    eta: 0:00:02  time: 0.0593  data_time: 0.0066  memory: 527  
03/19 21:54:45 - mmengine - INFO - Epoch(val)  [59][250/250]    eta: 0:00:00  time: 0.0567  data_time: 0.0064  memory: 527  
03/19 21:54:46 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.41s).
Accumulating evaluation results...
DONE (t=1.98s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.565
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.159
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.276
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.428
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.509
03/19 21:54:56 - mmengine - INFO - bbox_mAP_copypaste: 0.234 0.565 0.151 0.159 0.276 0.428
03/19 21:54:56 - mmengine - INFO - Epoch(val) [59][250/250]    coco/bbox_mAP: 0.2340  coco/bbox_mAP_50: 0.5650  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1590  coco/bbox_mAP_m: 0.2760  coco/bbox_mAP_l: 0.4280  data_time: 0.0067  time: 0.0588
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:55:08 - mmengine - INFO - Epoch(train)  [60][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:54  time: 0.2453  data_time: 0.0181  memory: 3937  loss: 4.4015  loss_cls: 0.6790  loss_bbox: 2.3574  loss_obj: 1.3652
03/19 21:55:19 - mmengine - INFO - Epoch(train)  [60][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:43  time: 0.2190  data_time: 0.0082  memory: 3071  loss: 4.3697  loss_cls: 0.6700  loss_bbox: 2.3419  loss_obj: 1.3578
03/19 21:55:29 - mmengine - INFO - Epoch(train)  [60][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:32  time: 0.2000  data_time: 0.0083  memory: 2587  loss: 4.3444  loss_cls: 0.6811  loss_bbox: 2.3340  loss_obj: 1.3293
03/19 21:55:40 - mmengine - INFO - Epoch(train)  [60][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:21  time: 0.2174  data_time: 0.0082  memory: 3639  loss: 4.3297  loss_cls: 0.6722  loss_bbox: 2.3415  loss_obj: 1.3160
03/19 21:55:51 - mmengine - INFO - Epoch(train)  [60][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:10  time: 0.2140  data_time: 0.0083  memory: 3639  loss: 4.3336  loss_cls: 0.6717  loss_bbox: 2.3343  loss_obj: 1.3275
03/19 21:56:02 - mmengine - INFO - Epoch(train)  [60][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:27:00  time: 0.2294  data_time: 0.0084  memory: 3937  loss: 4.3538  loss_cls: 0.6709  loss_bbox: 2.3465  loss_obj: 1.3365
03/19 21:56:13 - mmengine - INFO - Epoch(train)  [60][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:26:49  time: 0.2204  data_time: 0.0083  memory: 3937  loss: 4.3097  loss_cls: 0.6727  loss_bbox: 2.3237  loss_obj: 1.3132
03/19 21:56:25 - mmengine - INFO - Epoch(train)  [60][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:26:38  time: 0.2207  data_time: 0.0082  memory: 3937  loss: 4.3784  loss_cls: 0.6934  loss_bbox: 2.3550  loss_obj: 1.3300
03/19 21:56:36 - mmengine - INFO - Epoch(train)  [60][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:26:28  time: 0.2314  data_time: 0.0082  memory: 3639  loss: 4.3747  loss_cls: 0.6743  loss_bbox: 2.3587  loss_obj: 1.3418
03/19 21:56:47 - mmengine - INFO - Epoch(train)  [60][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:26:17  time: 0.2236  data_time: 0.0081  memory: 3937  loss: 4.3816  loss_cls: 0.6784  loss_bbox: 2.3440  loss_obj: 1.3593
03/19 21:57:00 - mmengine - INFO - Epoch(train)  [60][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:26:07  time: 0.2439  data_time: 0.0081  memory: 3937  loss: 4.3774  loss_cls: 0.6766  loss_bbox: 2.3545  loss_obj: 1.3463
03/19 21:57:10 - mmengine - INFO - Epoch(train)  [60][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:57  time: 0.2171  data_time: 0.0083  memory: 3357  loss: 4.3326  loss_cls: 0.6745  loss_bbox: 2.3515  loss_obj: 1.3065
03/19 21:57:21 - mmengine - INFO - Epoch(train)  [60][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:46  time: 0.2155  data_time: 0.0083  memory: 3639  loss: 4.3813  loss_cls: 0.6815  loss_bbox: 2.3515  loss_obj: 1.3483
03/19 21:57:31 - mmengine - INFO - Epoch(train)  [60][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:34  time: 0.1916  data_time: 0.0084  memory: 2817  loss: 4.4047  loss_cls: 0.6878  loss_bbox: 2.4013  loss_obj: 1.3156
03/19 21:57:42 - mmengine - INFO - Epoch(train)  [60][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:23  time: 0.2243  data_time: 0.0082  memory: 3639  loss: 4.3728  loss_cls: 0.6762  loss_bbox: 2.3405  loss_obj: 1.3561
03/19 21:57:54 - mmengine - INFO - Epoch(train)  [60][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:13  time: 0.2425  data_time: 0.0082  memory: 3937  loss: 4.3052  loss_cls: 0.6593  loss_bbox: 2.3080  loss_obj: 1.3379
03/19 21:58:05 - mmengine - INFO - Epoch(train)  [60][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:25:03  time: 0.2154  data_time: 0.0082  memory: 3937  loss: 4.3470  loss_cls: 0.6737  loss_bbox: 2.3486  loss_obj: 1.3247
03/19 21:58:15 - mmengine - INFO - Epoch(train)  [60][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:24:51  time: 0.2106  data_time: 0.0082  memory: 3357  loss: 4.2859  loss_cls: 0.6729  loss_bbox: 2.3246  loss_obj: 1.2884
03/19 21:58:25 - mmengine - INFO - Epoch(train)  [60][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:24:40  time: 0.1882  data_time: 0.0084  memory: 2587  loss: 4.4417  loss_cls: 0.6935  loss_bbox: 2.3956  loss_obj: 1.3526
03/19 21:58:35 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 21:58:35 - mmengine - INFO - Epoch(train)  [60][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:24:29  time: 0.2118  data_time: 0.0081  memory: 3937  loss: 4.4319  loss_cls: 0.6970  loss_bbox: 2.3833  loss_obj: 1.3516
03/19 21:58:36 - mmengine - INFO - Saving checkpoint at 60 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:58:41 - mmengine - INFO - Epoch(val)  [60][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0073  memory: 527  
03/19 21:58:44 - mmengine - INFO - Epoch(val)  [60][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0066  memory: 527  
03/19 21:58:47 - mmengine - INFO - Epoch(val)  [60][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0066  memory: 527  
03/19 21:58:50 - mmengine - INFO - Epoch(val)  [60][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 21:58:53 - mmengine - INFO - Epoch(val)  [60][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 21:58:54 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.51s).
Accumulating evaluation results...
DONE (t=1.99s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.567
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.152
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.160
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.273
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.442
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.277
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.385
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.514
03/19 21:59:04 - mmengine - INFO - bbox_mAP_copypaste: 0.234 0.567 0.152 0.160 0.273 0.442
03/19 21:59:04 - mmengine - INFO - Epoch(val) [60][250/250]    coco/bbox_mAP: 0.2340  coco/bbox_mAP_50: 0.5670  coco/bbox_mAP_75: 0.1520  coco/bbox_mAP_s: 0.1600  coco/bbox_mAP_m: 0.2730  coco/bbox_mAP_l: 0.4420  data_time: 0.0067  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 21:59:16 - mmengine - INFO - Epoch(train)  [61][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:24:18  time: 0.2332  data_time: 0.0183  memory: 3937  loss: 4.4224  loss_cls: 0.6919  loss_bbox: 2.3658  loss_obj: 1.3648
03/19 21:59:26 - mmengine - INFO - Epoch(train)  [61][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:24:07  time: 0.2124  data_time: 0.0084  memory: 3937  loss: 4.4496  loss_cls: 0.6892  loss_bbox: 2.3794  loss_obj: 1.3811
03/19 21:59:39 - mmengine - INFO - Epoch(train)  [61][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:57  time: 0.2430  data_time: 0.0081  memory: 3937  loss: 4.3747  loss_cls: 0.6749  loss_bbox: 2.3538  loss_obj: 1.3461
03/19 21:59:49 - mmengine - INFO - Epoch(train)  [61][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:46  time: 0.2068  data_time: 0.0081  memory: 3639  loss: 4.2793  loss_cls: 0.6720  loss_bbox: 2.3361  loss_obj: 1.2711
03/19 22:00:01 - mmengine - INFO - Epoch(train)  [61][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:36  time: 0.2333  data_time: 0.0082  memory: 3937  loss: 4.3450  loss_cls: 0.6767  loss_bbox: 2.3235  loss_obj: 1.3447
03/19 22:00:10 - mmengine - INFO - Epoch(train)  [61][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:25  time: 0.1973  data_time: 0.0084  memory: 3357  loss: 4.3689  loss_cls: 0.6928  loss_bbox: 2.3581  loss_obj: 1.3180
03/19 22:00:22 - mmengine - INFO - Epoch(train)  [61][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:14  time: 0.2361  data_time: 0.0083  memory: 3937  loss: 4.3425  loss_cls: 0.6766  loss_bbox: 2.3249  loss_obj: 1.3410
03/19 22:00:32 - mmengine - INFO - Epoch(train)  [61][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:23:03  time: 0.2038  data_time: 0.0082  memory: 2817  loss: 4.3833  loss_cls: 0.6884  loss_bbox: 2.3492  loss_obj: 1.3457
03/19 22:00:43 - mmengine - INFO - Epoch(train)  [61][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:22:52  time: 0.2152  data_time: 0.0082  memory: 3937  loss: 4.3714  loss_cls: 0.6891  loss_bbox: 2.3505  loss_obj: 1.3318
03/19 22:00:55 - mmengine - INFO - Epoch(train)  [61][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:22:42  time: 0.2267  data_time: 0.0081  memory: 3639  loss: 4.3128  loss_cls: 0.6696  loss_bbox: 2.3130  loss_obj: 1.3302
03/19 22:01:06 - mmengine - INFO - Epoch(train)  [61][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:22:31  time: 0.2278  data_time: 0.0083  memory: 3639  loss: 4.3161  loss_cls: 0.6764  loss_bbox: 2.3388  loss_obj: 1.3008
03/19 22:01:16 - mmengine - INFO - Epoch(train)  [61][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:22:20  time: 0.2014  data_time: 0.0083  memory: 3357  loss: 4.3220  loss_cls: 0.6819  loss_bbox: 2.3316  loss_obj: 1.3086
03/19 22:01:27 - mmengine - INFO - Epoch(train)  [61][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:22:09  time: 0.2108  data_time: 0.0081  memory: 3937  loss: 4.3243  loss_cls: 0.6783  loss_bbox: 2.3417  loss_obj: 1.3043
03/19 22:01:37 - mmengine - INFO - Epoch(train)  [61][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:58  time: 0.2133  data_time: 0.0081  memory: 3071  loss: 4.2634  loss_cls: 0.6658  loss_bbox: 2.3215  loss_obj: 1.2761
03/19 22:01:47 - mmengine - INFO - Epoch(train)  [61][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:47  time: 0.2027  data_time: 0.0084  memory: 3639  loss: 4.3424  loss_cls: 0.6761  loss_bbox: 2.3555  loss_obj: 1.3107
03/19 22:01:57 - mmengine - INFO - Epoch(train)  [61][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:35  time: 0.1969  data_time: 0.0082  memory: 3937  loss: 4.3486  loss_cls: 0.6832  loss_bbox: 2.3484  loss_obj: 1.3171
03/19 22:02:09 - mmengine - INFO - Epoch(train)  [61][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:25  time: 0.2319  data_time: 0.0083  memory: 3639  loss: 4.3741  loss_cls: 0.6788  loss_bbox: 2.3485  loss_obj: 1.3469
03/19 22:02:19 - mmengine - INFO - Epoch(train)  [61][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:13  time: 0.2032  data_time: 0.0083  memory: 3357  loss: 4.2845  loss_cls: 0.6722  loss_bbox: 2.3138  loss_obj: 1.2986
03/19 22:02:31 - mmengine - INFO - Epoch(train)  [61][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:21:03  time: 0.2406  data_time: 0.0082  memory: 3937  loss: 4.2946  loss_cls: 0.6585  loss_bbox: 2.3065  loss_obj: 1.3296
03/19 22:02:42 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:02:42 - mmengine - INFO - Epoch(train)  [61][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:20:53  time: 0.2235  data_time: 0.0081  memory: 3937  loss: 4.2852  loss_cls: 0.6746  loss_bbox: 2.3148  loss_obj: 1.2958
03/19 22:02:42 - mmengine - INFO - Saving checkpoint at 61 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:02:48 - mmengine - INFO - Epoch(val)  [61][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0072  memory: 527  
03/19 22:02:51 - mmengine - INFO - Epoch(val)  [61][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 22:02:54 - mmengine - INFO - Epoch(val)  [61][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 22:02:57 - mmengine - INFO - Epoch(val)  [61][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 22:02:59 - mmengine - INFO - Epoch(val)  [61][250/250]    eta: 0:00:00  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 22:03:01 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.34s).
Accumulating evaluation results...
DONE (t=2.24s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.565
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.152
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.160
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.273
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.450
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.335
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.335
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.335
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.278
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.384
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.516
03/19 22:03:11 - mmengine - INFO - bbox_mAP_copypaste: 0.234 0.565 0.152 0.160 0.273 0.450
03/19 22:03:11 - mmengine - INFO - Epoch(val) [61][250/250]    coco/bbox_mAP: 0.2340  coco/bbox_mAP_50: 0.5650  coco/bbox_mAP_75: 0.1520  coco/bbox_mAP_s: 0.1600  coco/bbox_mAP_m: 0.2730  coco/bbox_mAP_l: 0.4500  data_time: 0.0067  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:03:22 - mmengine - INFO - Epoch(train)  [62][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:20:42  time: 0.2189  data_time: 0.0190  memory: 3639  loss: 4.3424  loss_cls: 0.6832  loss_bbox: 2.3673  loss_obj: 1.2919
03/19 22:03:32 - mmengine - INFO - Epoch(train)  [62][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:20:31  time: 0.1961  data_time: 0.0083  memory: 2587  loss: 4.3359  loss_cls: 0.6879  loss_bbox: 2.3634  loss_obj: 1.2846
03/19 22:03:42 - mmengine - INFO - Epoch(train)  [62][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:20:19  time: 0.2085  data_time: 0.0082  memory: 3639  loss: 4.3437  loss_cls: 0.6764  loss_bbox: 2.3359  loss_obj: 1.3314
03/19 22:03:53 - mmengine - INFO - Epoch(train)  [62][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:20:08  time: 0.2065  data_time: 0.0082  memory: 3357  loss: 4.2439  loss_cls: 0.6595  loss_bbox: 2.3127  loss_obj: 1.2716
03/19 22:04:03 - mmengine - INFO - Epoch(train)  [62][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:57  time: 0.2119  data_time: 0.0084  memory: 3937  loss: 4.3151  loss_cls: 0.6764  loss_bbox: 2.3499  loss_obj: 1.2888
03/19 22:04:14 - mmengine - INFO - Epoch(train)  [62][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:46  time: 0.2130  data_time: 0.0082  memory: 3639  loss: 4.3680  loss_cls: 0.6752  loss_bbox: 2.3459  loss_obj: 1.3470
03/19 22:04:25 - mmengine - INFO - Epoch(train)  [62][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:36  time: 0.2267  data_time: 0.0082  memory: 3937  loss: 4.2983  loss_cls: 0.6689  loss_bbox: 2.3135  loss_obj: 1.3159
03/19 22:04:37 - mmengine - INFO - Epoch(train)  [62][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:25  time: 0.2237  data_time: 0.0083  memory: 3937  loss: 4.3551  loss_cls: 0.6766  loss_bbox: 2.3512  loss_obj: 1.3272
03/19 22:04:48 - mmengine - INFO - Epoch(train)  [62][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:15  time: 0.2362  data_time: 0.0082  memory: 3639  loss: 4.3109  loss_cls: 0.6676  loss_bbox: 2.3102  loss_obj: 1.3332
03/19 22:04:59 - mmengine - INFO - Epoch(train)  [62][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:19:04  time: 0.2052  data_time: 0.0084  memory: 3937  loss: 4.3181  loss_cls: 0.6781  loss_bbox: 2.3441  loss_obj: 1.2959
03/19 22:05:09 - mmengine - INFO - Epoch(train)  [62][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:18:53  time: 0.2130  data_time: 0.0083  memory: 3639  loss: 4.2531  loss_cls: 0.6695  loss_bbox: 2.3143  loss_obj: 1.2693
03/19 22:05:20 - mmengine - INFO - Epoch(train)  [62][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:18:42  time: 0.2080  data_time: 0.0083  memory: 3639  loss: 4.3815  loss_cls: 0.6859  loss_bbox: 2.3665  loss_obj: 1.3290
03/19 22:05:30 - mmengine - INFO - Epoch(train)  [62][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:18:31  time: 0.2124  data_time: 0.0083  memory: 3639  loss: 4.4016  loss_cls: 0.6880  loss_bbox: 2.3778  loss_obj: 1.3357
03/19 22:05:42 - mmengine - INFO - Epoch(train)  [62][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:18:20  time: 0.2339  data_time: 0.0082  memory: 3639  loss: 4.3050  loss_cls: 0.6659  loss_bbox: 2.3220  loss_obj: 1.3171
03/19 22:05:54 - mmengine - INFO - Epoch(train)  [62][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:18:10  time: 0.2440  data_time: 0.0082  memory: 3937  loss: 4.3022  loss_cls: 0.6608  loss_bbox: 2.3036  loss_obj: 1.3379
03/19 22:06:05 - mmengine - INFO - Epoch(train)  [62][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:59  time: 0.2101  data_time: 0.0083  memory: 3937  loss: 4.3299  loss_cls: 0.6734  loss_bbox: 2.3395  loss_obj: 1.3169
03/19 22:06:17 - mmengine - INFO - Epoch(train)  [62][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:49  time: 0.2394  data_time: 0.0083  memory: 3937  loss: 4.3618  loss_cls: 0.6775  loss_bbox: 2.3430  loss_obj: 1.3413
03/19 22:06:28 - mmengine - INFO - Epoch(train)  [62][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:39  time: 0.2219  data_time: 0.0082  memory: 3639  loss: 4.2728  loss_cls: 0.6635  loss_bbox: 2.2873  loss_obj: 1.3221
03/19 22:06:38 - mmengine - INFO - Epoch(train)  [62][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:27  time: 0.2022  data_time: 0.0082  memory: 3357  loss: 4.3598  loss_cls: 0.6826  loss_bbox: 2.3453  loss_obj: 1.3319
03/19 22:06:49 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:06:49 - mmengine - INFO - Epoch(train)  [62][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:17  time: 0.2204  data_time: 0.0081  memory: 3639  loss: 4.3618  loss_cls: 0.6742  loss_bbox: 2.3337  loss_obj: 1.3539
03/19 22:06:49 - mmengine - INFO - Saving checkpoint at 62 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:06:54 - mmengine - INFO - Epoch(val)  [62][ 50/250]    eta: 0:00:11  time: 0.0583  data_time: 0.0071  memory: 527  
03/19 22:06:57 - mmengine - INFO - Epoch(val)  [62][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 22:07:00 - mmengine - INFO - Epoch(val)  [62][150/250]    eta: 0:00:05  time: 0.0595  data_time: 0.0066  memory: 527  
03/19 22:07:03 - mmengine - INFO - Epoch(val)  [62][200/250]    eta: 0:00:02  time: 0.0590  data_time: 0.0066  memory: 527  
03/19 22:07:06 - mmengine - INFO - Epoch(val)  [62][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0064  memory: 527  
03/19 22:07:07 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.31s).
Accumulating evaluation results...
DONE (t=2.23s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.235
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.570
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.274
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.452
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.383
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.518
03/19 22:07:18 - mmengine - INFO - bbox_mAP_copypaste: 0.235 0.570 0.149 0.161 0.274 0.452
03/19 22:07:18 - mmengine - INFO - Epoch(val) [62][250/250]    coco/bbox_mAP: 0.2350  coco/bbox_mAP_50: 0.5700  coco/bbox_mAP_75: 0.1490  coco/bbox_mAP_s: 0.1610  coco/bbox_mAP_m: 0.2740  coco/bbox_mAP_l: 0.4520  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:07:29 - mmengine - INFO - Epoch(train)  [63][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:17:06  time: 0.2314  data_time: 0.0188  memory: 3937  loss: 4.4053  loss_cls: 0.6834  loss_bbox: 2.3584  loss_obj: 1.3634
03/19 22:07:40 - mmengine - INFO - Epoch(train)  [63][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:56  time: 0.2251  data_time: 0.0082  memory: 3639  loss: 4.3288  loss_cls: 0.6734  loss_bbox: 2.3304  loss_obj: 1.3250
03/19 22:07:51 - mmengine - INFO - Epoch(train)  [63][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:44  time: 0.2060  data_time: 0.0083  memory: 3071  loss: 4.3527  loss_cls: 0.6800  loss_bbox: 2.3475  loss_obj: 1.3252
03/19 22:08:02 - mmengine - INFO - Epoch(train)  [63][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:34  time: 0.2323  data_time: 0.0081  memory: 3639  loss: 4.2696  loss_cls: 0.6568  loss_bbox: 2.2916  loss_obj: 1.3212
03/19 22:08:13 - mmengine - INFO - Epoch(train)  [63][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:23  time: 0.2214  data_time: 0.0083  memory: 3937  loss: 4.3701  loss_cls: 0.6844  loss_bbox: 2.3414  loss_obj: 1.3443
03/19 22:08:24 - mmengine - INFO - Epoch(train)  [63][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:12  time: 0.2096  data_time: 0.0083  memory: 3071  loss: 4.3090  loss_cls: 0.6726  loss_bbox: 2.3446  loss_obj: 1.2918
03/19 22:08:36 - mmengine - INFO - Epoch(train)  [63][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:16:02  time: 0.2392  data_time: 0.0082  memory: 3937  loss: 4.3675  loss_cls: 0.6723  loss_bbox: 2.3409  loss_obj: 1.3543
03/19 22:08:48 - mmengine - INFO - Epoch(train)  [63][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:15:52  time: 0.2450  data_time: 0.0082  memory: 3937  loss: 4.3201  loss_cls: 0.6726  loss_bbox: 2.3312  loss_obj: 1.3164
03/19 22:08:58 - mmengine - INFO - Epoch(train)  [63][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:15:41  time: 0.2016  data_time: 0.0084  memory: 3937  loss: 4.3742  loss_cls: 0.6885  loss_bbox: 2.3747  loss_obj: 1.3110
03/19 22:09:09 - mmengine - INFO - Epoch(train)  [63][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:15:30  time: 0.2048  data_time: 0.0083  memory: 3071  loss: 4.3573  loss_cls: 0.6830  loss_bbox: 2.3625  loss_obj: 1.3118
03/19 22:09:19 - mmengine - INFO - Epoch(train)  [63][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:15:18  time: 0.2046  data_time: 0.0083  memory: 3071  loss: 4.3661  loss_cls: 0.6798  loss_bbox: 2.3519  loss_obj: 1.3344
03/19 22:09:29 - mmengine - INFO - Epoch(train)  [63][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:15:07  time: 0.1970  data_time: 0.0082  memory: 2817  loss: 4.3610  loss_cls: 0.6783  loss_bbox: 2.3441  loss_obj: 1.3386
03/19 22:09:39 - mmengine - INFO - Epoch(train)  [63][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:56  time: 0.2125  data_time: 0.0082  memory: 3639  loss: 4.3560  loss_cls: 0.6809  loss_bbox: 2.3456  loss_obj: 1.3296
03/19 22:09:50 - mmengine - INFO - Epoch(train)  [63][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:45  time: 0.2033  data_time: 0.0085  memory: 3937  loss: 4.3194  loss_cls: 0.6732  loss_bbox: 2.3448  loss_obj: 1.3014
03/19 22:09:59 - mmengine - INFO - Epoch(train)  [63][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:33  time: 0.1955  data_time: 0.0083  memory: 3639  loss: 4.3663  loss_cls: 0.6940  loss_bbox: 2.3686  loss_obj: 1.3038
03/19 22:10:10 - mmengine - INFO - Epoch(train)  [63][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:22  time: 0.2095  data_time: 0.0084  memory: 3639  loss: 4.3877  loss_cls: 0.6841  loss_bbox: 2.3603  loss_obj: 1.3433
03/19 22:10:20 - mmengine - INFO - Epoch(train)  [63][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:11  time: 0.2113  data_time: 0.0081  memory: 2817  loss: 4.2827  loss_cls: 0.6611  loss_bbox: 2.3282  loss_obj: 1.2935
03/19 22:10:31 - mmengine - INFO - Epoch(train)  [63][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:14:01  time: 0.2209  data_time: 0.0081  memory: 3639  loss: 4.3213  loss_cls: 0.6683  loss_bbox: 2.3241  loss_obj: 1.3289
03/19 22:10:42 - mmengine - INFO - Epoch(train)  [63][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:13:49  time: 0.2036  data_time: 0.0081  memory: 3071  loss: 4.3226  loss_cls: 0.6748  loss_bbox: 2.3395  loss_obj: 1.3083
03/19 22:10:53 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:10:53 - mmengine - INFO - Epoch(train)  [63][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:13:39  time: 0.2185  data_time: 0.0081  memory: 3937  loss: 4.3415  loss_cls: 0.6689  loss_bbox: 2.3390  loss_obj: 1.3335
03/19 22:10:53 - mmengine - INFO - Saving checkpoint at 63 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:10:58 - mmengine - INFO - Epoch(val)  [63][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 22:11:01 - mmengine - INFO - Epoch(val)  [63][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 22:11:04 - mmengine - INFO - Epoch(val)  [63][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0064  memory: 527  
03/19 22:11:07 - mmengine - INFO - Epoch(val)  [63][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 22:11:10 - mmengine - INFO - Epoch(val)  [63][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 22:11:11 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.34s).
Accumulating evaluation results...
DONE (t=2.23s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.236
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.570
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.152
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.162
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.275
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.471
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.337
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.337
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.337
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.384
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.519
03/19 22:11:21 - mmengine - INFO - bbox_mAP_copypaste: 0.236 0.570 0.152 0.162 0.275 0.471
03/19 22:11:21 - mmengine - INFO - Epoch(val) [63][250/250]    coco/bbox_mAP: 0.2360  coco/bbox_mAP_50: 0.5700  coco/bbox_mAP_75: 0.1520  coco/bbox_mAP_s: 0.1620  coco/bbox_mAP_m: 0.2750  coco/bbox_mAP_l: 0.4710  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:11:32 - mmengine - INFO - Epoch(train)  [64][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:13:28  time: 0.2111  data_time: 0.0190  memory: 2817  loss: 4.3152  loss_cls: 0.6664  loss_bbox: 2.3492  loss_obj: 1.2996
03/19 22:11:42 - mmengine - INFO - Epoch(train)  [64][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:13:16  time: 0.2066  data_time: 0.0083  memory: 3357  loss: 4.3357  loss_cls: 0.6737  loss_bbox: 2.3475  loss_obj: 1.3145
03/19 22:11:53 - mmengine - INFO - Epoch(train)  [64][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:13:06  time: 0.2185  data_time: 0.0084  memory: 3937  loss: 4.4181  loss_cls: 0.6868  loss_bbox: 2.3759  loss_obj: 1.3554
03/19 22:12:04 - mmengine - INFO - Epoch(train)  [64][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:55  time: 0.2161  data_time: 0.0082  memory: 3937  loss: 4.3182  loss_cls: 0.6690  loss_bbox: 2.3414  loss_obj: 1.3079
03/19 22:12:14 - mmengine - INFO - Epoch(train)  [64][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:43  time: 0.2014  data_time: 0.0083  memory: 3357  loss: 4.3709  loss_cls: 0.6802  loss_bbox: 2.3609  loss_obj: 1.3297
03/19 22:12:25 - mmengine - INFO - Epoch(train)  [64][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:33  time: 0.2300  data_time: 0.0083  memory: 3937  loss: 4.3021  loss_cls: 0.6696  loss_bbox: 2.3083  loss_obj: 1.3243
03/19 22:12:37 - mmengine - INFO - Epoch(train)  [64][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:23  time: 0.2284  data_time: 0.0082  memory: 3639  loss: 4.3225  loss_cls: 0.6728  loss_bbox: 2.3332  loss_obj: 1.3164
03/19 22:12:47 - mmengine - INFO - Epoch(train)  [64][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:12  time: 0.2106  data_time: 0.0083  memory: 3357  loss: 4.2699  loss_cls: 0.6724  loss_bbox: 2.3005  loss_obj: 1.2970
03/19 22:12:59 - mmengine - INFO - Epoch(train)  [64][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:12:01  time: 0.2239  data_time: 0.0081  memory: 3357  loss: 4.3463  loss_cls: 0.6833  loss_bbox: 2.3326  loss_obj: 1.3304
03/19 22:13:10 - mmengine - INFO - Epoch(train)  [64][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:11:51  time: 0.2360  data_time: 0.0081  memory: 3937  loss: 4.3467  loss_cls: 0.6698  loss_bbox: 2.3355  loss_obj: 1.3415
03/19 22:13:22 - mmengine - INFO - Epoch(train)  [64][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:11:40  time: 0.2389  data_time: 0.0083  memory: 3937  loss: 4.2920  loss_cls: 0.6756  loss_bbox: 2.3193  loss_obj: 1.2971
03/19 22:13:32 - mmengine - INFO - Epoch(train)  [64][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:11:29  time: 0.1962  data_time: 0.0082  memory: 3071  loss: 4.3143  loss_cls: 0.6827  loss_bbox: 2.3477  loss_obj: 1.2839
03/19 22:13:43 - mmengine - INFO - Epoch(train)  [64][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:11:18  time: 0.2107  data_time: 0.0083  memory: 3357  loss: 4.2568  loss_cls: 0.6700  loss_bbox: 2.3130  loss_obj: 1.2737
03/19 22:13:54 - mmengine - INFO - Epoch(train)  [64][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:11:07  time: 0.2248  data_time: 0.0082  memory: 3357  loss: 4.3576  loss_cls: 0.6804  loss_bbox: 2.3223  loss_obj: 1.3549
03/19 22:14:05 - mmengine - INFO - Epoch(train)  [64][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:57  time: 0.2169  data_time: 0.0082  memory: 3357  loss: 4.2811  loss_cls: 0.6637  loss_bbox: 2.3239  loss_obj: 1.2936
03/19 22:14:15 - mmengine - INFO - Epoch(train)  [64][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:45  time: 0.2047  data_time: 0.0083  memory: 3639  loss: 4.3441  loss_cls: 0.6735  loss_bbox: 2.3466  loss_obj: 1.3240
03/19 22:14:27 - mmengine - INFO - Epoch(train)  [64][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:35  time: 0.2374  data_time: 0.0081  memory: 3937  loss: 4.3640  loss_cls: 0.6842  loss_bbox: 2.3481  loss_obj: 1.3317
03/19 22:14:38 - mmengine - INFO - Epoch(train)  [64][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:24  time: 0.2108  data_time: 0.0083  memory: 3357  loss: 4.3657  loss_cls: 0.6873  loss_bbox: 2.3442  loss_obj: 1.3342
03/19 22:14:49 - mmengine - INFO - Epoch(train)  [64][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:13  time: 0.2208  data_time: 0.0082  memory: 3357  loss: 4.3940  loss_cls: 0.6797  loss_bbox: 2.3656  loss_obj: 1.3487
03/19 22:14:58 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:14:58 - mmengine - INFO - Epoch(train)  [64][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:10:02  time: 0.1968  data_time: 0.0082  memory: 3071  loss: 4.3341  loss_cls: 0.6821  loss_bbox: 2.3351  loss_obj: 1.3170
03/19 22:14:58 - mmengine - INFO - Saving checkpoint at 64 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:15:04 - mmengine - INFO - Epoch(val)  [64][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0071  memory: 527  
03/19 22:15:07 - mmengine - INFO - Epoch(val)  [64][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0064  memory: 527  
03/19 22:15:10 - mmengine - INFO - Epoch(val)  [64][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 22:15:13 - mmengine - INFO - Epoch(val)  [64][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 22:15:15 - mmengine - INFO - Epoch(val)  [64][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0065  memory: 527  
03/19 22:15:17 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.24s).
Accumulating evaluation results...
DONE (t=1.97s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.236
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.571
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.163
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.277
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.464
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.511
03/19 22:15:27 - mmengine - INFO - bbox_mAP_copypaste: 0.236 0.571 0.150 0.163 0.277 0.464
03/19 22:15:27 - mmengine - INFO - Epoch(val) [64][250/250]    coco/bbox_mAP: 0.2360  coco/bbox_mAP_50: 0.5710  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1630  coco/bbox_mAP_m: 0.2770  coco/bbox_mAP_l: 0.4640  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:15:39 - mmengine - INFO - Epoch(train)  [65][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:09:52  time: 0.2484  data_time: 0.0188  memory: 3937  loss: 4.3087  loss_cls: 0.6741  loss_bbox: 2.3049  loss_obj: 1.3296
03/19 22:15:49 - mmengine - INFO - Epoch(train)  [65][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:09:41  time: 0.2008  data_time: 0.0083  memory: 2817  loss: 4.3038  loss_cls: 0.6756  loss_bbox: 2.3480  loss_obj: 1.2802
03/19 22:16:01 - mmengine - INFO - Epoch(train)  [65][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:09:30  time: 0.2324  data_time: 0.0081  memory: 3357  loss: 4.3265  loss_cls: 0.6653  loss_bbox: 2.3173  loss_obj: 1.3439
03/19 22:16:12 - mmengine - INFO - Epoch(train)  [65][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:09:19  time: 0.2142  data_time: 0.0082  memory: 3937  loss: 4.3052  loss_cls: 0.6709  loss_bbox: 2.3261  loss_obj: 1.3081
03/19 22:16:24 - mmengine - INFO - Epoch(train)  [65][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:09:10  time: 0.2509  data_time: 0.0083  memory: 3937  loss: 4.3866  loss_cls: 0.6638  loss_bbox: 2.3451  loss_obj: 1.3776
03/19 22:16:34 - mmengine - INFO - Epoch(train)  [65][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:58  time: 0.1999  data_time: 0.0082  memory: 2817  loss: 4.2682  loss_cls: 0.6705  loss_bbox: 2.3287  loss_obj: 1.2690
03/19 22:16:45 - mmengine - INFO - Epoch(train)  [65][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:47  time: 0.2184  data_time: 0.0082  memory: 3071  loss: 4.2756  loss_cls: 0.6666  loss_bbox: 2.3230  loss_obj: 1.2860
03/19 22:16:55 - mmengine - INFO - Epoch(train)  [65][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:36  time: 0.2035  data_time: 0.0082  memory: 3071  loss: 4.3540  loss_cls: 0.6848  loss_bbox: 2.3359  loss_obj: 1.3333
03/19 22:17:06 - mmengine - INFO - Epoch(train)  [65][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:26  time: 0.2237  data_time: 0.0083  memory: 3639  loss: 4.3535  loss_cls: 0.6719  loss_bbox: 2.3449  loss_obj: 1.3367
03/19 22:17:16 - mmengine - INFO - Epoch(train)  [65][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:14  time: 0.1992  data_time: 0.0082  memory: 3357  loss: 4.3364  loss_cls: 0.6691  loss_bbox: 2.3497  loss_obj: 1.3177
03/19 22:17:27 - mmengine - INFO - Epoch(train)  [65][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:08:03  time: 0.2068  data_time: 0.0082  memory: 2817  loss: 4.3634  loss_cls: 0.6766  loss_bbox: 2.3393  loss_obj: 1.3476
03/19 22:17:38 - mmengine - INFO - Epoch(train)  [65][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:07:53  time: 0.2279  data_time: 0.0082  memory: 3937  loss: 4.3035  loss_cls: 0.6627  loss_bbox: 2.3160  loss_obj: 1.3248
03/19 22:17:50 - mmengine - INFO - Epoch(train)  [65][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:07:42  time: 0.2382  data_time: 0.0081  memory: 3937  loss: 4.2878  loss_cls: 0.6680  loss_bbox: 2.3167  loss_obj: 1.3032
03/19 22:18:01 - mmengine - INFO - Epoch(train)  [65][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:07:32  time: 0.2250  data_time: 0.0082  memory: 3937  loss: 4.3024  loss_cls: 0.6656  loss_bbox: 2.3187  loss_obj: 1.3180
03/19 22:18:12 - mmengine - INFO - Epoch(train)  [65][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:07:21  time: 0.2167  data_time: 0.0081  memory: 3357  loss: 4.3415  loss_cls: 0.6774  loss_bbox: 2.3578  loss_obj: 1.3064
03/19 22:18:21 - mmengine - INFO - Epoch(train)  [65][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:07:09  time: 0.1829  data_time: 0.0085  memory: 3071  loss: 4.3603  loss_cls: 0.6821  loss_bbox: 2.3799  loss_obj: 1.2982
03/19 22:18:33 - mmengine - INFO - Epoch(train)  [65][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:59  time: 0.2268  data_time: 0.0082  memory: 3937  loss: 4.3225  loss_cls: 0.6727  loss_bbox: 2.3124  loss_obj: 1.3374
03/19 22:18:44 - mmengine - INFO - Epoch(train)  [65][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:48  time: 0.2181  data_time: 0.0082  memory: 3357  loss: 4.3536  loss_cls: 0.6787  loss_bbox: 2.3227  loss_obj: 1.3522
03/19 22:18:55 - mmengine - INFO - Epoch(train)  [65][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:37  time: 0.2181  data_time: 0.0084  memory: 3937  loss: 4.3478  loss_cls: 0.6739  loss_bbox: 2.3275  loss_obj: 1.3464
03/19 22:19:06 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:19:06 - mmengine - INFO - Epoch(train)  [65][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:26  time: 0.2196  data_time: 0.0082  memory: 3357  loss: 4.3853  loss_cls: 0.6788  loss_bbox: 2.3347  loss_obj: 1.3719
03/19 22:19:06 - mmengine - INFO - Saving checkpoint at 65 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:19:11 - mmengine - INFO - Epoch(val)  [65][ 50/250]    eta: 0:00:11  time: 0.0583  data_time: 0.0071  memory: 527  
03/19 22:19:14 - mmengine - INFO - Epoch(val)  [65][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0065  memory: 527  
03/19 22:19:17 - mmengine - INFO - Epoch(val)  [65][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 22:19:20 - mmengine - INFO - Epoch(val)  [65][200/250]    eta: 0:00:02  time: 0.0591  data_time: 0.0065  memory: 527  
03/19 22:19:22 - mmengine - INFO - Epoch(val)  [65][250/250]    eta: 0:00:00  time: 0.0567  data_time: 0.0064  memory: 527  
03/19 22:19:24 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.47s).
Accumulating evaluation results...
DONE (t=1.98s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.236
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.575
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.162
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.286
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.469
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.336
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.385
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.516
03/19 22:19:34 - mmengine - INFO - bbox_mAP_copypaste: 0.236 0.575 0.151 0.162 0.286 0.469
03/19 22:19:34 - mmengine - INFO - Epoch(val) [65][250/250]    coco/bbox_mAP: 0.2360  coco/bbox_mAP_50: 0.5750  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1620  coco/bbox_mAP_m: 0.2860  coco/bbox_mAP_l: 0.4690  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:19:45 - mmengine - INFO - Epoch(train)  [66][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:16  time: 0.2250  data_time: 0.0190  memory: 3639  loss: 4.3264  loss_cls: 0.6750  loss_bbox: 2.3374  loss_obj: 1.3140
03/19 22:19:56 - mmengine - INFO - Epoch(train)  [66][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:06:05  time: 0.2205  data_time: 0.0081  memory: 3639  loss: 4.3714  loss_cls: 0.6749  loss_bbox: 2.3478  loss_obj: 1.3488
03/19 22:20:07 - mmengine - INFO - Epoch(train)  [66][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:05:54  time: 0.2081  data_time: 0.0083  memory: 3639  loss: 4.2933  loss_cls: 0.6644  loss_bbox: 2.3306  loss_obj: 1.2983
03/19 22:20:18 - mmengine - INFO - Epoch(train)  [66][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:05:43  time: 0.2274  data_time: 0.0085  memory: 3937  loss: 4.3827  loss_cls: 0.6799  loss_bbox: 2.3499  loss_obj: 1.3529
03/19 22:20:28 - mmengine - INFO - Epoch(train)  [66][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:05:32  time: 0.1987  data_time: 0.0082  memory: 2587  loss: 4.2739  loss_cls: 0.6750  loss_bbox: 2.3149  loss_obj: 1.2840
03/19 22:20:39 - mmengine - INFO - Epoch(train)  [66][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:05:21  time: 0.2147  data_time: 0.0082  memory: 3937  loss: 4.3346  loss_cls: 0.6693  loss_bbox: 2.3475  loss_obj: 1.3178
03/19 22:20:50 - mmengine - INFO - Epoch(train)  [66][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:05:10  time: 0.2166  data_time: 0.0083  memory: 3071  loss: 4.3249  loss_cls: 0.6729  loss_bbox: 2.3483  loss_obj: 1.3038
03/19 22:20:59 - mmengine - INFO - Epoch(train)  [66][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:58  time: 0.1799  data_time: 0.0085  memory: 2337  loss: 4.3406  loss_cls: 0.6822  loss_bbox: 2.3687  loss_obj: 1.2896
03/19 22:21:09 - mmengine - INFO - Epoch(train)  [66][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:47  time: 0.2090  data_time: 0.0082  memory: 3357  loss: 4.3960  loss_cls: 0.6790  loss_bbox: 2.3541  loss_obj: 1.3629
03/19 22:21:20 - mmengine - INFO - Epoch(train)  [66][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:37  time: 0.2194  data_time: 0.0082  memory: 3357  loss: 4.2701  loss_cls: 0.6714  loss_bbox: 2.2994  loss_obj: 1.2993
03/19 22:21:31 - mmengine - INFO - Epoch(train)  [66][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:26  time: 0.2196  data_time: 0.0084  memory: 3639  loss: 4.3078  loss_cls: 0.6654  loss_bbox: 2.3216  loss_obj: 1.3208
03/19 22:21:42 - mmengine - INFO - Epoch(train)  [66][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:15  time: 0.2164  data_time: 0.0083  memory: 3639  loss: 4.2950  loss_cls: 0.6674  loss_bbox: 2.3237  loss_obj: 1.3039
03/19 22:21:53 - mmengine - INFO - Epoch(train)  [66][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:04:04  time: 0.2192  data_time: 0.0082  memory: 3071  loss: 4.3183  loss_cls: 0.6618  loss_bbox: 2.3146  loss_obj: 1.3419
03/19 22:22:04 - mmengine - INFO - Epoch(train)  [66][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:03:54  time: 0.2298  data_time: 0.0082  memory: 3639  loss: 4.3308  loss_cls: 0.6672  loss_bbox: 2.3239  loss_obj: 1.3397
03/19 22:22:15 - mmengine - INFO - Epoch(train)  [66][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:03:43  time: 0.2169  data_time: 0.0084  memory: 3639  loss: 4.2904  loss_cls: 0.6740  loss_bbox: 2.3099  loss_obj: 1.3065
03/19 22:22:26 - mmengine - INFO - Epoch(train)  [66][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:03:32  time: 0.2242  data_time: 0.0082  memory: 3937  loss: 4.3118  loss_cls: 0.6623  loss_bbox: 2.3299  loss_obj: 1.3196
03/19 22:22:37 - mmengine - INFO - Epoch(train)  [66][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:03:21  time: 0.2126  data_time: 0.0082  memory: 3937  loss: 4.2894  loss_cls: 0.6662  loss_bbox: 2.3177  loss_obj: 1.3054
03/19 22:22:47 - mmengine - INFO - Epoch(train)  [66][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:03:10  time: 0.2024  data_time: 0.0083  memory: 3639  loss: 4.3382  loss_cls: 0.6860  loss_bbox: 2.3560  loss_obj: 1.2961
03/19 22:22:58 - mmengine - INFO - Epoch(train)  [66][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:59  time: 0.2063  data_time: 0.0082  memory: 3071  loss: 4.3715  loss_cls: 0.6821  loss_bbox: 2.3475  loss_obj: 1.3419
03/19 22:23:09 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:23:09 - mmengine - INFO - Epoch(train)  [66][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:48  time: 0.2199  data_time: 0.0080  memory: 3357  loss: 4.3806  loss_cls: 0.6796  loss_bbox: 2.3422  loss_obj: 1.3589
03/19 22:23:09 - mmengine - INFO - Saving checkpoint at 66 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:23:14 - mmengine - INFO - Epoch(val)  [66][ 50/250]    eta: 0:00:11  time: 0.0588  data_time: 0.0071  memory: 527  
03/19 22:23:17 - mmengine - INFO - Epoch(val)  [66][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0064  memory: 527  
03/19 22:23:20 - mmengine - INFO - Epoch(val)  [66][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 22:23:23 - mmengine - INFO - Epoch(val)  [66][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 22:23:26 - mmengine - INFO - Epoch(val)  [66][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0064  memory: 527  
03/19 22:23:27 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.43s).
Accumulating evaluation results...
DONE (t=1.97s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.237
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.575
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.288
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.468
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.395
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.518
03/19 22:23:37 - mmengine - INFO - bbox_mAP_copypaste: 0.237 0.575 0.151 0.161 0.288 0.468
03/19 22:23:37 - mmengine - INFO - Epoch(val) [66][250/250]    coco/bbox_mAP: 0.2370  coco/bbox_mAP_50: 0.5750  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1610  coco/bbox_mAP_m: 0.2880  coco/bbox_mAP_l: 0.4680  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:23:48 - mmengine - INFO - Epoch(train)  [67][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:37  time: 0.2135  data_time: 0.0187  memory: 3639  loss: 4.2544  loss_cls: 0.6693  loss_bbox: 2.3095  loss_obj: 1.2756
03/19 22:23:57 - mmengine - INFO - Epoch(train)  [67][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:26  time: 0.1940  data_time: 0.0083  memory: 2587  loss: 4.3155  loss_cls: 0.6779  loss_bbox: 2.3380  loss_obj: 1.2996
03/19 22:24:09 - mmengine - INFO - Epoch(train)  [67][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:15  time: 0.2230  data_time: 0.0082  memory: 3937  loss: 4.3417  loss_cls: 0.6751  loss_bbox: 2.3442  loss_obj: 1.3224
03/19 22:24:20 - mmengine - INFO - Epoch(train)  [67][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:02:05  time: 0.2296  data_time: 0.0081  memory: 3937  loss: 4.3162  loss_cls: 0.6674  loss_bbox: 2.3123  loss_obj: 1.3365
03/19 22:24:32 - mmengine - INFO - Epoch(train)  [67][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:01:54  time: 0.2332  data_time: 0.0083  memory: 3937  loss: 4.3590  loss_cls: 0.6719  loss_bbox: 2.3377  loss_obj: 1.3494
03/19 22:24:42 - mmengine - INFO - Epoch(train)  [67][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:01:43  time: 0.1976  data_time: 0.0081  memory: 2337  loss: 4.3478  loss_cls: 0.6739  loss_bbox: 2.3507  loss_obj: 1.3232
03/19 22:24:53 - mmengine - INFO - Epoch(train)  [67][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:01:32  time: 0.2196  data_time: 0.0081  memory: 3937  loss: 4.3956  loss_cls: 0.6771  loss_bbox: 2.3738  loss_obj: 1.3447
03/19 22:25:03 - mmengine - INFO - Epoch(train)  [67][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:01:21  time: 0.2042  data_time: 0.0084  memory: 3357  loss: 4.3535  loss_cls: 0.6858  loss_bbox: 2.3458  loss_obj: 1.3219
03/19 22:25:14 - mmengine - INFO - Epoch(train)  [67][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:01:11  time: 0.2296  data_time: 0.0082  memory: 3639  loss: 4.3363  loss_cls: 0.6722  loss_bbox: 2.3165  loss_obj: 1.3476
03/19 22:25:25 - mmengine - INFO - Epoch(train)  [67][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:59  time: 0.2042  data_time: 0.0083  memory: 3071  loss: 4.2883  loss_cls: 0.6653  loss_bbox: 2.3178  loss_obj: 1.3052
03/19 22:25:34 - mmengine - INFO - Epoch(train)  [67][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:48  time: 0.1968  data_time: 0.0082  memory: 2817  loss: 4.3051  loss_cls: 0.6731  loss_bbox: 2.3430  loss_obj: 1.2890
03/19 22:25:46 - mmengine - INFO - Epoch(train)  [67][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:37  time: 0.2243  data_time: 0.0080  memory: 3937  loss: 4.2956  loss_cls: 0.6625  loss_bbox: 2.3212  loss_obj: 1.3119
03/19 22:25:55 - mmengine - INFO - Epoch(train)  [67][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:26  time: 0.1902  data_time: 0.0089  memory: 3071  loss: 4.3409  loss_cls: 0.6872  loss_bbox: 2.3512  loss_obj: 1.3025
03/19 22:26:06 - mmengine - INFO - Epoch(train)  [67][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:15  time: 0.2241  data_time: 0.0082  memory: 3937  loss: 4.3651  loss_cls: 0.6769  loss_bbox: 2.3482  loss_obj: 1.3400
03/19 22:26:16 - mmengine - INFO - Epoch(train)  [67][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 2:00:04  time: 0.1948  data_time: 0.0085  memory: 3639  loss: 4.3490  loss_cls: 0.6837  loss_bbox: 2.3574  loss_obj: 1.3079
03/19 22:26:28 - mmengine - INFO - Epoch(train)  [67][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:59:53  time: 0.2308  data_time: 0.0082  memory: 3357  loss: 4.2980  loss_cls: 0.6630  loss_bbox: 2.3025  loss_obj: 1.3325
03/19 22:26:39 - mmengine - INFO - Epoch(train)  [67][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:59:43  time: 0.2201  data_time: 0.0083  memory: 3639  loss: 4.3498  loss_cls: 0.6702  loss_bbox: 2.3298  loss_obj: 1.3498
03/19 22:26:49 - mmengine - INFO - Epoch(train)  [67][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:59:32  time: 0.2137  data_time: 0.0083  memory: 3937  loss: 4.3386  loss_cls: 0.6763  loss_bbox: 2.3574  loss_obj: 1.3049
03/19 22:27:01 - mmengine - INFO - Epoch(train)  [67][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:59:21  time: 0.2259  data_time: 0.0083  memory: 3639  loss: 4.3258  loss_cls: 0.6767  loss_bbox: 2.3318  loss_obj: 1.3173
03/19 22:27:11 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:27:11 - mmengine - INFO - Epoch(train)  [67][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:59:10  time: 0.1975  data_time: 0.0082  memory: 2817  loss: 4.2954  loss_cls: 0.6747  loss_bbox: 2.3426  loss_obj: 1.2781
03/19 22:27:11 - mmengine - INFO - Saving checkpoint at 67 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:27:16 - mmengine - INFO - Epoch(val)  [67][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0071  memory: 527  
03/19 22:27:19 - mmengine - INFO - Epoch(val)  [67][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 22:27:22 - mmengine - INFO - Epoch(val)  [67][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 22:27:25 - mmengine - INFO - Epoch(val)  [67][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 22:27:28 - mmengine - INFO - Epoch(val)  [67][250/250]    eta: 0:00:00  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 22:27:29 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.49s).
Accumulating evaluation results...
DONE (t=1.97s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.235
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.565
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.159
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.290
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.469
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.282
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.398
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.519
03/19 22:27:39 - mmengine - INFO - bbox_mAP_copypaste: 0.235 0.565 0.151 0.159 0.290 0.469
03/19 22:27:39 - mmengine - INFO - Epoch(val) [67][250/250]    coco/bbox_mAP: 0.2350  coco/bbox_mAP_50: 0.5650  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1590  coco/bbox_mAP_m: 0.2900  coco/bbox_mAP_l: 0.4690  data_time: 0.0066  time: 0.0587
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:27:50 - mmengine - INFO - Epoch(train)  [68][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:59  time: 0.2109  data_time: 0.0190  memory: 3071  loss: 4.2647  loss_cls: 0.6650  loss_bbox: 2.3293  loss_obj: 1.2704
03/19 22:28:01 - mmengine - INFO - Epoch(train)  [68][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:48  time: 0.2247  data_time: 0.0082  memory: 3357  loss: 4.3218  loss_cls: 0.6794  loss_bbox: 2.3369  loss_obj: 1.3055
03/19 22:28:12 - mmengine - INFO - Epoch(train)  [68][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:37  time: 0.2091  data_time: 0.0081  memory: 3357  loss: 4.2767  loss_cls: 0.6676  loss_bbox: 2.3078  loss_obj: 1.3013
03/19 22:28:22 - mmengine - INFO - Epoch(train)  [68][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:26  time: 0.2059  data_time: 0.0083  memory: 3937  loss: 4.3213  loss_cls: 0.6812  loss_bbox: 2.3462  loss_obj: 1.2939
03/19 22:28:32 - mmengine - INFO - Epoch(train)  [68][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:15  time: 0.2112  data_time: 0.0083  memory: 3937  loss: 4.2674  loss_cls: 0.6666  loss_bbox: 2.3334  loss_obj: 1.2674
03/19 22:28:42 - mmengine - INFO - Epoch(train)  [68][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:58:04  time: 0.1976  data_time: 0.0084  memory: 3357  loss: 4.3146  loss_cls: 0.6758  loss_bbox: 2.3577  loss_obj: 1.2810
03/19 22:28:55 - mmengine - INFO - Epoch(train)  [68][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:57:54  time: 0.2589  data_time: 0.0081  memory: 3937  loss: 4.3917  loss_cls: 0.6770  loss_bbox: 2.3230  loss_obj: 1.3916
03/19 22:29:06 - mmengine - INFO - Epoch(train)  [68][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:57:43  time: 0.2153  data_time: 0.0082  memory: 3639  loss: 4.3389  loss_cls: 0.6852  loss_bbox: 2.3355  loss_obj: 1.3182
03/19 22:29:17 - mmengine - INFO - Epoch(train)  [68][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:57:33  time: 0.2283  data_time: 0.0083  memory: 3937  loss: 4.3325  loss_cls: 0.6681  loss_bbox: 2.3334  loss_obj: 1.3310
03/19 22:29:28 - mmengine - INFO - Epoch(train)  [68][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:57:21  time: 0.2004  data_time: 0.0082  memory: 3357  loss: 4.3581  loss_cls: 0.6874  loss_bbox: 2.3593  loss_obj: 1.3115
03/19 22:29:38 - mmengine - INFO - Epoch(train)  [68][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:57:10  time: 0.2036  data_time: 0.0082  memory: 3071  loss: 4.2841  loss_cls: 0.6685  loss_bbox: 2.3141  loss_obj: 1.3014
03/19 22:29:49 - mmengine - INFO - Epoch(train)  [68][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:59  time: 0.2167  data_time: 0.0082  memory: 3639  loss: 4.3524  loss_cls: 0.6770  loss_bbox: 2.3500  loss_obj: 1.3254
03/19 22:29:57 - mmengine - INFO - Epoch(train)  [68][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:47  time: 0.1714  data_time: 0.0086  memory: 1690  loss: 4.3138  loss_cls: 0.6827  loss_bbox: 2.3711  loss_obj: 1.2600
03/19 22:30:09 - mmengine - INFO - Epoch(train)  [68][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:37  time: 0.2329  data_time: 0.0083  memory: 3937  loss: 4.3618  loss_cls: 0.6795  loss_bbox: 2.3495  loss_obj: 1.3328
03/19 22:30:20 - mmengine - INFO - Epoch(train)  [68][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:26  time: 0.2207  data_time: 0.0081  memory: 3639  loss: 4.3260  loss_cls: 0.6707  loss_bbox: 2.3443  loss_obj: 1.3111
03/19 22:30:31 - mmengine - INFO - Epoch(train)  [68][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:16  time: 0.2260  data_time: 0.0081  memory: 3937  loss: 4.2644  loss_cls: 0.6632  loss_bbox: 2.3082  loss_obj: 1.2930
03/19 22:30:41 - mmengine - INFO - Epoch(train)  [68][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:56:05  time: 0.2041  data_time: 0.0082  memory: 3357  loss: 4.3839  loss_cls: 0.6805  loss_bbox: 2.3730  loss_obj: 1.3305
03/19 22:30:53 - mmengine - INFO - Epoch(train)  [68][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:54  time: 0.2231  data_time: 0.0081  memory: 3357  loss: 4.3287  loss_cls: 0.6786  loss_bbox: 2.3190  loss_obj: 1.3311
03/19 22:31:04 - mmengine - INFO - Epoch(train)  [68][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:43  time: 0.2238  data_time: 0.0082  memory: 3639  loss: 4.3028  loss_cls: 0.6602  loss_bbox: 2.3059  loss_obj: 1.3367
03/19 22:31:16 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:31:16 - mmengine - INFO - Epoch(train)  [68][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:33  time: 0.2416  data_time: 0.0081  memory: 3937  loss: 4.3095  loss_cls: 0.6743  loss_bbox: 2.2964  loss_obj: 1.3388
03/19 22:31:16 - mmengine - INFO - Saving checkpoint at 68 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:31:21 - mmengine - INFO - Epoch(val)  [68][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0072  memory: 527  
03/19 22:31:24 - mmengine - INFO - Epoch(val)  [68][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 22:31:27 - mmengine - INFO - Epoch(val)  [68][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 22:31:30 - mmengine - INFO - Epoch(val)  [68][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 22:31:33 - mmengine - INFO - Epoch(val)  [68][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 22:31:34 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.27s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.47s).
Accumulating evaluation results...
DONE (t=1.96s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.565
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.157
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.288
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.470
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.520
03/19 22:31:44 - mmengine - INFO - bbox_mAP_copypaste: 0.234 0.565 0.151 0.157 0.288 0.470
03/19 22:31:44 - mmengine - INFO - Epoch(val) [68][250/250]    coco/bbox_mAP: 0.2340  coco/bbox_mAP_50: 0.5650  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1570  coco/bbox_mAP_m: 0.2880  coco/bbox_mAP_l: 0.4700  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:31:57 - mmengine - INFO - Epoch(train)  [69][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:23  time: 0.2484  data_time: 0.0192  memory: 3937  loss: 4.3988  loss_cls: 0.6765  loss_bbox: 2.3655  loss_obj: 1.3568
03/19 22:32:07 - mmengine - INFO - Epoch(train)  [69][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:12  time: 0.1991  data_time: 0.0083  memory: 3071  loss: 4.3728  loss_cls: 0.6779  loss_bbox: 2.3703  loss_obj: 1.3246
03/19 22:32:18 - mmengine - INFO - Epoch(train)  [69][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:55:01  time: 0.2314  data_time: 0.0082  memory: 3937  loss: 4.2982  loss_cls: 0.6703  loss_bbox: 2.3056  loss_obj: 1.3222
03/19 22:32:29 - mmengine - INFO - Epoch(train)  [69][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:54:50  time: 0.2226  data_time: 0.0081  memory: 3937  loss: 4.2585  loss_cls: 0.6636  loss_bbox: 2.3114  loss_obj: 1.2836
03/19 22:32:41 - mmengine - INFO - Epoch(train)  [69][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:54:40  time: 0.2255  data_time: 0.0082  memory: 3937  loss: 4.3744  loss_cls: 0.6759  loss_bbox: 2.3627  loss_obj: 1.3359
03/19 22:32:52 - mmengine - INFO - Epoch(train)  [69][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:54:29  time: 0.2271  data_time: 0.0081  memory: 3639  loss: 4.2773  loss_cls: 0.6739  loss_bbox: 2.3052  loss_obj: 1.2981
03/19 22:33:02 - mmengine - INFO - Epoch(train)  [69][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:54:18  time: 0.1934  data_time: 0.0083  memory: 3071  loss: 4.3318  loss_cls: 0.6836  loss_bbox: 2.3519  loss_obj: 1.2963
03/19 22:33:11 - mmengine - INFO - Epoch(train)  [69][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:54:06  time: 0.1921  data_time: 0.0082  memory: 2587  loss: 4.2530  loss_cls: 0.6791  loss_bbox: 2.3121  loss_obj: 1.2617
03/19 22:33:21 - mmengine - INFO - Epoch(train)  [69][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:55  time: 0.2006  data_time: 0.0082  memory: 3071  loss: 4.2883  loss_cls: 0.6674  loss_bbox: 2.3186  loss_obj: 1.3023
03/19 22:33:32 - mmengine - INFO - Epoch(train)  [69][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:44  time: 0.2074  data_time: 0.0083  memory: 3639  loss: 4.3189  loss_cls: 0.6745  loss_bbox: 2.3231  loss_obj: 1.3213
03/19 22:33:42 - mmengine - INFO - Epoch(train)  [69][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:33  time: 0.2008  data_time: 0.0083  memory: 3071  loss: 4.3440  loss_cls: 0.6823  loss_bbox: 2.3561  loss_obj: 1.3057
03/19 22:33:52 - mmengine - INFO - Epoch(train)  [69][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:22  time: 0.2074  data_time: 0.0083  memory: 3937  loss: 4.2819  loss_cls: 0.6807  loss_bbox: 2.3283  loss_obj: 1.2729
03/19 22:34:03 - mmengine - INFO - Epoch(train)  [69][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:11  time: 0.2219  data_time: 0.0082  memory: 3639  loss: 4.2838  loss_cls: 0.6687  loss_bbox: 2.3182  loss_obj: 1.2969
03/19 22:34:14 - mmengine - INFO - Epoch(train)  [69][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:53:00  time: 0.2170  data_time: 0.0082  memory: 3639  loss: 4.3562  loss_cls: 0.6832  loss_bbox: 2.3495  loss_obj: 1.3235
03/19 22:34:25 - mmengine - INFO - Epoch(train)  [69][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:52:50  time: 0.2130  data_time: 0.0083  memory: 3937  loss: 4.3414  loss_cls: 0.6789  loss_bbox: 2.3350  loss_obj: 1.3276
03/19 22:34:34 - mmengine - INFO - Epoch(train)  [69][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:52:38  time: 0.1904  data_time: 0.0083  memory: 2587  loss: 4.2999  loss_cls: 0.6648  loss_bbox: 2.3411  loss_obj: 1.2940
03/19 22:34:45 - mmengine - INFO - Epoch(train)  [69][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:52:27  time: 0.2127  data_time: 0.0083  memory: 3071  loss: 4.3298  loss_cls: 0.6767  loss_bbox: 2.3406  loss_obj: 1.3125
03/19 22:34:55 - mmengine - INFO - Epoch(train)  [69][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:52:16  time: 0.2024  data_time: 0.0082  memory: 3357  loss: 4.3498  loss_cls: 0.6762  loss_bbox: 2.3527  loss_obj: 1.3209
03/19 22:35:06 - mmengine - INFO - Epoch(train)  [69][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:52:05  time: 0.2132  data_time: 0.0083  memory: 3937  loss: 4.3332  loss_cls: 0.6701  loss_bbox: 2.3429  loss_obj: 1.3202
03/19 22:35:17 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:35:17 - mmengine - INFO - Epoch(train)  [69][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:55  time: 0.2284  data_time: 0.0081  memory: 3639  loss: 4.3197  loss_cls: 0.6714  loss_bbox: 2.3361  loss_obj: 1.3122
03/19 22:35:17 - mmengine - INFO - Saving checkpoint at 69 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:35:23 - mmengine - INFO - Epoch(val)  [69][ 50/250]    eta: 0:00:11  time: 0.0598  data_time: 0.0072  memory: 527  
03/19 22:35:26 - mmengine - INFO - Epoch(val)  [69][100/250]    eta: 0:00:08  time: 0.0580  data_time: 0.0064  memory: 527  
03/19 22:35:28 - mmengine - INFO - Epoch(val)  [69][150/250]    eta: 0:00:05  time: 0.0582  data_time: 0.0064  memory: 527  
03/19 22:35:31 - mmengine - INFO - Epoch(val)  [69][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0064  memory: 527  
03/19 22:35:34 - mmengine - INFO - Epoch(val)  [69][250/250]    eta: 0:00:00  time: 0.0565  data_time: 0.0064  memory: 527  
03/19 22:35:36 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.47s).
Accumulating evaluation results...
DONE (t=1.97s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.562
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.157
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.290
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.463
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.281
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.526
03/19 22:35:46 - mmengine - INFO - bbox_mAP_copypaste: 0.234 0.562 0.150 0.157 0.290 0.463
03/19 22:35:46 - mmengine - INFO - Epoch(val) [69][250/250]    coco/bbox_mAP: 0.2340  coco/bbox_mAP_50: 0.5620  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1570  coco/bbox_mAP_m: 0.2900  coco/bbox_mAP_l: 0.4630  data_time: 0.0066  time: 0.0581
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:35:57 - mmengine - INFO - Epoch(train)  [70][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:44  time: 0.2331  data_time: 0.0185  memory: 3937  loss: 4.3302  loss_cls: 0.6752  loss_bbox: 2.3189  loss_obj: 1.3361
03/19 22:36:08 - mmengine - INFO - Epoch(train)  [70][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:33  time: 0.2057  data_time: 0.0084  memory: 3639  loss: 4.4069  loss_cls: 0.6822  loss_bbox: 2.3689  loss_obj: 1.3558
03/19 22:36:19 - mmengine - INFO - Epoch(train)  [70][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:22  time: 0.2241  data_time: 0.0083  memory: 3639  loss: 4.3245  loss_cls: 0.6701  loss_bbox: 2.3487  loss_obj: 1.3057
03/19 22:36:29 - mmengine - INFO - Epoch(train)  [70][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:11  time: 0.2076  data_time: 0.0082  memory: 3639  loss: 4.3656  loss_cls: 0.6903  loss_bbox: 2.3602  loss_obj: 1.3152
03/19 22:36:41 - mmengine - INFO - Epoch(train)  [70][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:51:01  time: 0.2383  data_time: 0.0083  memory: 3639  loss: 4.3000  loss_cls: 0.6707  loss_bbox: 2.3118  loss_obj: 1.3175
03/19 22:36:53 - mmengine - INFO - Epoch(train)  [70][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:50:50  time: 0.2329  data_time: 0.0082  memory: 3937  loss: 4.3202  loss_cls: 0.6726  loss_bbox: 2.3327  loss_obj: 1.3149
03/19 22:37:04 - mmengine - INFO - Epoch(train)  [70][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:50:40  time: 0.2134  data_time: 0.0083  memory: 3357  loss: 4.3338  loss_cls: 0.6727  loss_bbox: 2.3384  loss_obj: 1.3226
03/19 22:37:15 - mmengine - INFO - Epoch(train)  [70][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:50:29  time: 0.2264  data_time: 0.0083  memory: 3937  loss: 4.3292  loss_cls: 0.6786  loss_bbox: 2.3277  loss_obj: 1.3228
03/19 22:37:26 - mmengine - INFO - Epoch(train)  [70][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:50:18  time: 0.2213  data_time: 0.0082  memory: 3639  loss: 4.2795  loss_cls: 0.6655  loss_bbox: 2.3140  loss_obj: 1.3000
03/19 22:37:39 - mmengine - INFO - Epoch(train)  [70][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:50:08  time: 0.2622  data_time: 0.0082  memory: 3937  loss: 4.1936  loss_cls: 0.6458  loss_bbox: 2.2628  loss_obj: 1.2851
03/19 22:37:49 - mmengine - INFO - Epoch(train)  [70][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:57  time: 0.2014  data_time: 0.0083  memory: 3357  loss: 4.3426  loss_cls: 0.6870  loss_bbox: 2.3522  loss_obj: 1.3035
03/19 22:38:00 - mmengine - INFO - Epoch(train)  [70][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:46  time: 0.2088  data_time: 0.0082  memory: 3078  loss: 4.3029  loss_cls: 0.6762  loss_bbox: 2.3326  loss_obj: 1.2941
03/19 22:38:10 - mmengine - INFO - Epoch(train)  [70][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:35  time: 0.2114  data_time: 0.0081  memory: 2817  loss: 4.3005  loss_cls: 0.6662  loss_bbox: 2.3479  loss_obj: 1.2865
03/19 22:38:21 - mmengine - INFO - Epoch(train)  [70][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:24  time: 0.2052  data_time: 0.0083  memory: 3357  loss: 4.2262  loss_cls: 0.6653  loss_bbox: 2.3137  loss_obj: 1.2473
03/19 22:38:31 - mmengine - INFO - Epoch(train)  [70][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:13  time: 0.2060  data_time: 0.0084  memory: 3639  loss: 4.3786  loss_cls: 0.6820  loss_bbox: 2.3424  loss_obj: 1.3542
03/19 22:38:42 - mmengine - INFO - Epoch(train)  [70][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:49:02  time: 0.2207  data_time: 0.0081  memory: 3639  loss: 4.3365  loss_cls: 0.6703  loss_bbox: 2.3495  loss_obj: 1.3167
03/19 22:38:53 - mmengine - INFO - Epoch(train)  [70][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:48:52  time: 0.2269  data_time: 0.0082  memory: 3639  loss: 4.2701  loss_cls: 0.6664  loss_bbox: 2.2953  loss_obj: 1.3084
03/19 22:39:03 - mmengine - INFO - Epoch(train)  [70][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:48:41  time: 0.2025  data_time: 0.0085  memory: 3357  loss: 4.3497  loss_cls: 0.6801  loss_bbox: 2.3568  loss_obj: 1.3128
03/19 22:39:14 - mmengine - INFO - Epoch(train)  [70][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:48:30  time: 0.2177  data_time: 0.0083  memory: 3937  loss: 4.3576  loss_cls: 0.6783  loss_bbox: 2.3515  loss_obj: 1.3277
03/19 22:39:24 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:39:24 - mmengine - INFO - Epoch(train)  [70][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:48:19  time: 0.1967  data_time: 0.0082  memory: 3357  loss: 4.3086  loss_cls: 0.6775  loss_bbox: 2.3408  loss_obj: 1.2902
03/19 22:39:24 - mmengine - INFO - Saving checkpoint at 70 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:39:29 - mmengine - INFO - Epoch(val)  [70][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0072  memory: 527  
03/19 22:39:32 - mmengine - INFO - Epoch(val)  [70][100/250]    eta: 0:00:08  time: 0.0590  data_time: 0.0064  memory: 527  
03/19 22:39:35 - mmengine - INFO - Epoch(val)  [70][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 22:39:38 - mmengine - INFO - Epoch(val)  [70][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0064  memory: 527  
03/19 22:39:41 - mmengine - INFO - Epoch(val)  [70][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 22:39:42 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.42s).
Accumulating evaluation results...
DONE (t=1.96s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.235
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.563
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.152
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.156
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.294
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.462
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.341
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.341
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.341
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.405
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.525
03/19 22:39:52 - mmengine - INFO - bbox_mAP_copypaste: 0.235 0.563 0.152 0.156 0.294 0.462
03/19 22:39:52 - mmengine - INFO - Epoch(val) [70][250/250]    coco/bbox_mAP: 0.2350  coco/bbox_mAP_50: 0.5630  coco/bbox_mAP_75: 0.1520  coco/bbox_mAP_s: 0.1560  coco/bbox_mAP_m: 0.2940  coco/bbox_mAP_l: 0.4620  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:40:03 - mmengine - INFO - Epoch(train)  [71][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:48:08  time: 0.2132  data_time: 0.0186  memory: 3357  loss: 4.3010  loss_cls: 0.6839  loss_bbox: 2.3506  loss_obj: 1.2666
03/19 22:40:16 - mmengine - INFO - Epoch(train)  [71][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:58  time: 0.2534  data_time: 0.0082  memory: 3937  loss: 4.3454  loss_cls: 0.6596  loss_bbox: 2.3080  loss_obj: 1.3778
03/19 22:40:26 - mmengine - INFO - Epoch(train)  [71][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:47  time: 0.2088  data_time: 0.0081  memory: 3639  loss: 4.3777  loss_cls: 0.6706  loss_bbox: 2.3725  loss_obj: 1.3347
03/19 22:40:37 - mmengine - INFO - Epoch(train)  [71][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:36  time: 0.2085  data_time: 0.0084  memory: 3357  loss: 4.3297  loss_cls: 0.6787  loss_bbox: 2.3539  loss_obj: 1.2970
03/19 22:40:48 - mmengine - INFO - Epoch(train)  [71][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:25  time: 0.2220  data_time: 0.0083  memory: 3937  loss: 4.2769  loss_cls: 0.6749  loss_bbox: 2.3257  loss_obj: 1.2763
03/19 22:40:57 - mmengine - INFO - Epoch(train)  [71][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:13  time: 0.1831  data_time: 0.0084  memory: 2587  loss: 4.3399  loss_cls: 0.6827  loss_bbox: 2.3530  loss_obj: 1.3043
03/19 22:41:08 - mmengine - INFO - Epoch(train)  [71][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:47:03  time: 0.2187  data_time: 0.0082  memory: 3071  loss: 4.2637  loss_cls: 0.6689  loss_bbox: 2.3072  loss_obj: 1.2875
03/19 22:41:18 - mmengine - INFO - Epoch(train)  [71][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:46:51  time: 0.2013  data_time: 0.0084  memory: 3937  loss: 4.2996  loss_cls: 0.6701  loss_bbox: 2.3284  loss_obj: 1.3011
03/19 22:41:28 - mmengine - INFO - Epoch(train)  [71][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:46:40  time: 0.2081  data_time: 0.0084  memory: 3937  loss: 4.3050  loss_cls: 0.6728  loss_bbox: 2.3431  loss_obj: 1.2891
03/19 22:41:40 - mmengine - INFO - Epoch(train)  [71][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:46:30  time: 0.2329  data_time: 0.0082  memory: 3639  loss: 4.2942  loss_cls: 0.6661  loss_bbox: 2.3411  loss_obj: 1.2869
03/19 22:41:50 - mmengine - INFO - Epoch(train)  [71][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:46:19  time: 0.2004  data_time: 0.0085  memory: 3071  loss: 4.2832  loss_cls: 0.6738  loss_bbox: 2.3327  loss_obj: 1.2766
03/19 22:42:02 - mmengine - INFO - Epoch(train)  [71][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:46:08  time: 0.2305  data_time: 0.0083  memory: 3937  loss: 4.3148  loss_cls: 0.6708  loss_bbox: 2.3098  loss_obj: 1.3342
03/19 22:42:12 - mmengine - INFO - Epoch(train)  [71][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:57  time: 0.2008  data_time: 0.0081  memory: 2587  loss: 4.3023  loss_cls: 0.6695  loss_bbox: 2.3164  loss_obj: 1.3164
03/19 22:42:22 - mmengine - INFO - Epoch(train)  [71][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:46  time: 0.1948  data_time: 0.0083  memory: 3639  loss: 4.2690  loss_cls: 0.6710  loss_bbox: 2.3261  loss_obj: 1.2718
03/19 22:42:33 - mmengine - INFO - Epoch(train)  [71][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:35  time: 0.2243  data_time: 0.0082  memory: 3639  loss: 4.2502  loss_cls: 0.6597  loss_bbox: 2.3105  loss_obj: 1.2800
03/19 22:42:44 - mmengine - INFO - Epoch(train)  [71][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:24  time: 0.2274  data_time: 0.0081  memory: 3937  loss: 4.3403  loss_cls: 0.6674  loss_bbox: 2.3406  loss_obj: 1.3323
03/19 22:42:54 - mmengine - INFO - Epoch(train)  [71][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:13  time: 0.2036  data_time: 0.0082  memory: 3639  loss: 4.2963  loss_cls: 0.6790  loss_bbox: 2.3221  loss_obj: 1.2953
03/19 22:43:06 - mmengine - INFO - Epoch(train)  [71][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:45:03  time: 0.2353  data_time: 0.0083  memory: 3937  loss: 4.3208  loss_cls: 0.6605  loss_bbox: 2.3269  loss_obj: 1.3334
03/19 22:43:17 - mmengine - INFO - Epoch(train)  [71][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:44:52  time: 0.2183  data_time: 0.0082  memory: 3937  loss: 4.3769  loss_cls: 0.6796  loss_bbox: 2.3625  loss_obj: 1.3348
03/19 22:43:29 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:43:29 - mmengine - INFO - Epoch(train)  [71][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:44:42  time: 0.2367  data_time: 0.0080  memory: 3937  loss: 4.2931  loss_cls: 0.6653  loss_bbox: 2.3000  loss_obj: 1.3278
03/19 22:43:29 - mmengine - INFO - Saving checkpoint at 71 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:43:34 - mmengine - INFO - Epoch(val)  [71][ 50/250]    eta: 0:00:11  time: 0.0585  data_time: 0.0073  memory: 527  
03/19 22:43:37 - mmengine - INFO - Epoch(val)  [71][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 22:43:40 - mmengine - INFO - Epoch(val)  [71][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 22:43:43 - mmengine - INFO - Epoch(val)  [71][200/250]    eta: 0:00:02  time: 0.0588  data_time: 0.0065  memory: 527  
03/19 22:43:46 - mmengine - INFO - Epoch(val)  [71][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 22:43:47 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.37s).
Accumulating evaluation results...
DONE (t=1.94s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.236
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.569
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.159
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.293
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.455
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.289
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.399
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.513
03/19 22:43:57 - mmengine - INFO - bbox_mAP_copypaste: 0.236 0.569 0.150 0.159 0.293 0.455
03/19 22:43:57 - mmengine - INFO - Epoch(val) [71][250/250]    coco/bbox_mAP: 0.2360  coco/bbox_mAP_50: 0.5690  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1590  coco/bbox_mAP_m: 0.2930  coco/bbox_mAP_l: 0.4550  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:44:07 - mmengine - INFO - Epoch(train)  [72][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:44:31  time: 0.2044  data_time: 0.0192  memory: 3071  loss: 4.2730  loss_cls: 0.6742  loss_bbox: 2.3205  loss_obj: 1.2783
03/19 22:44:20 - mmengine - INFO - Epoch(train)  [72][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:44:20  time: 0.2440  data_time: 0.0082  memory: 3937  loss: 4.3691  loss_cls: 0.6658  loss_bbox: 2.3277  loss_obj: 1.3757
03/19 22:44:31 - mmengine - INFO - Epoch(train)  [72][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:44:10  time: 0.2237  data_time: 0.0083  memory: 3639  loss: 4.3583  loss_cls: 0.6750  loss_bbox: 2.3414  loss_obj: 1.3419
03/19 22:44:41 - mmengine - INFO - Epoch(train)  [72][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:59  time: 0.2105  data_time: 0.0082  memory: 3071  loss: 4.3328  loss_cls: 0.6769  loss_bbox: 2.3461  loss_obj: 1.3097
03/19 22:44:52 - mmengine - INFO - Epoch(train)  [72][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:48  time: 0.2098  data_time: 0.0082  memory: 3357  loss: 4.3613  loss_cls: 0.6694  loss_bbox: 2.3288  loss_obj: 1.3632
03/19 22:45:02 - mmengine - INFO - Epoch(train)  [72][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:37  time: 0.2108  data_time: 0.0083  memory: 3639  loss: 4.3378  loss_cls: 0.6823  loss_bbox: 2.3537  loss_obj: 1.3018
03/19 22:45:13 - mmengine - INFO - Epoch(train)  [72][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:26  time: 0.2217  data_time: 0.0083  memory: 3937  loss: 4.2982  loss_cls: 0.6717  loss_bbox: 2.3335  loss_obj: 1.2929
03/19 22:45:24 - mmengine - INFO - Epoch(train)  [72][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:15  time: 0.2108  data_time: 0.0081  memory: 3639  loss: 4.3198  loss_cls: 0.6775  loss_bbox: 2.3471  loss_obj: 1.2951
03/19 22:45:36 - mmengine - INFO - Epoch(train)  [72][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:43:05  time: 0.2325  data_time: 0.0082  memory: 3639  loss: 4.3423  loss_cls: 0.6755  loss_bbox: 2.3218  loss_obj: 1.3450
03/19 22:45:46 - mmengine - INFO - Epoch(train)  [72][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:54  time: 0.2121  data_time: 0.0083  memory: 3639  loss: 4.2749  loss_cls: 0.6670  loss_bbox: 2.3255  loss_obj: 1.2824
03/19 22:45:58 - mmengine - INFO - Epoch(train)  [72][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:43  time: 0.2322  data_time: 0.0083  memory: 3937  loss: 4.2958  loss_cls: 0.6712  loss_bbox: 2.3040  loss_obj: 1.3206
03/19 22:46:09 - mmengine - INFO - Epoch(train)  [72][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:32  time: 0.2225  data_time: 0.0083  memory: 3357  loss: 4.2816  loss_cls: 0.6768  loss_bbox: 2.3364  loss_obj: 1.2685
03/19 22:46:20 - mmengine - INFO - Epoch(train)  [72][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:21  time: 0.2098  data_time: 0.0081  memory: 3071  loss: 4.2215  loss_cls: 0.6586  loss_bbox: 2.2901  loss_obj: 1.2727
03/19 22:46:31 - mmengine - INFO - Epoch(train)  [72][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:11  time: 0.2280  data_time: 0.0082  memory: 3937  loss: 4.2796  loss_cls: 0.6668  loss_bbox: 2.3314  loss_obj: 1.2814
03/19 22:46:41 - mmengine - INFO - Epoch(train)  [72][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:42:00  time: 0.2024  data_time: 0.0082  memory: 3071  loss: 4.3927  loss_cls: 0.6881  loss_bbox: 2.3606  loss_obj: 1.3440
03/19 22:46:53 - mmengine - INFO - Epoch(train)  [72][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:41:49  time: 0.2352  data_time: 0.0083  memory: 3937  loss: 4.3134  loss_cls: 0.6738  loss_bbox: 2.3348  loss_obj: 1.3048
03/19 22:47:04 - mmengine - INFO - Epoch(train)  [72][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:41:38  time: 0.2192  data_time: 0.0082  memory: 3639  loss: 4.2674  loss_cls: 0.6652  loss_bbox: 2.3152  loss_obj: 1.2870
03/19 22:47:14 - mmengine - INFO - Epoch(train)  [72][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:41:27  time: 0.2024  data_time: 0.0084  memory: 3357  loss: 4.3331  loss_cls: 0.6775  loss_bbox: 2.3466  loss_obj: 1.3090
03/19 22:47:26 - mmengine - INFO - Epoch(train)  [72][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:41:17  time: 0.2314  data_time: 0.0084  memory: 3937  loss: 4.3216  loss_cls: 0.6682  loss_bbox: 2.3341  loss_obj: 1.3192
03/19 22:47:36 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:47:36 - mmengine - INFO - Epoch(train)  [72][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:41:06  time: 0.2099  data_time: 0.0080  memory: 3937  loss: 4.2595  loss_cls: 0.6682  loss_bbox: 2.3105  loss_obj: 1.2807
03/19 22:47:36 - mmengine - INFO - Saving checkpoint at 72 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:47:41 - mmengine - INFO - Epoch(val)  [72][ 50/250]    eta: 0:00:11  time: 0.0590  data_time: 0.0072  memory: 527  
03/19 22:47:44 - mmengine - INFO - Epoch(val)  [72][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 22:47:47 - mmengine - INFO - Epoch(val)  [72][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 22:47:50 - mmengine - INFO - Epoch(val)  [72][200/250]    eta: 0:00:02  time: 0.0576  data_time: 0.0064  memory: 527  
03/19 22:47:53 - mmengine - INFO - Epoch(val)  [72][250/250]    eta: 0:00:00  time: 0.0577  data_time: 0.0065  memory: 527  
03/19 22:47:54 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.40s).
Accumulating evaluation results...
DONE (t=1.94s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.237
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.571
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.160
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.289
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.456
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.287
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.398
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.515
03/19 22:48:04 - mmengine - INFO - bbox_mAP_copypaste: 0.237 0.571 0.150 0.160 0.289 0.456
03/19 22:48:04 - mmengine - INFO - Epoch(val) [72][250/250]    coco/bbox_mAP: 0.2370  coco/bbox_mAP_50: 0.5710  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1600  coco/bbox_mAP_m: 0.2890  coco/bbox_mAP_l: 0.4560  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:48:16 - mmengine - INFO - Epoch(train)  [73][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:55  time: 0.2285  data_time: 0.0190  memory: 3639  loss: 4.3404  loss_cls: 0.6853  loss_bbox: 2.3258  loss_obj: 1.3293
03/19 22:48:27 - mmengine - INFO - Epoch(train)  [73][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:44  time: 0.2169  data_time: 0.0085  memory: 3639  loss: 4.2689  loss_cls: 0.6695  loss_bbox: 2.3113  loss_obj: 1.2881
03/19 22:48:38 - mmengine - INFO - Epoch(train)  [73][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:34  time: 0.2169  data_time: 0.0083  memory: 3639  loss: 4.2320  loss_cls: 0.6594  loss_bbox: 2.2938  loss_obj: 1.2789
03/19 22:48:48 - mmengine - INFO - Epoch(train)  [73][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:23  time: 0.2051  data_time: 0.0083  memory: 3071  loss: 4.2645  loss_cls: 0.6664  loss_bbox: 2.3129  loss_obj: 1.2853
03/19 22:48:58 - mmengine - INFO - Epoch(train)  [73][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:11  time: 0.1933  data_time: 0.0083  memory: 2587  loss: 4.2947  loss_cls: 0.6817  loss_bbox: 2.3326  loss_obj: 1.2804
03/19 22:49:08 - mmengine - INFO - Epoch(train)  [73][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:40:00  time: 0.2066  data_time: 0.0084  memory: 3639  loss: 4.2355  loss_cls: 0.6637  loss_bbox: 2.2969  loss_obj: 1.2748
03/19 22:49:18 - mmengine - INFO - Epoch(train)  [73][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:39:49  time: 0.1985  data_time: 0.0084  memory: 3357  loss: 4.3852  loss_cls: 0.6803  loss_bbox: 2.3736  loss_obj: 1.3313
03/19 22:49:29 - mmengine - INFO - Epoch(train)  [73][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:39:38  time: 0.2138  data_time: 0.0083  memory: 3357  loss: 4.2960  loss_cls: 0.6642  loss_bbox: 2.3250  loss_obj: 1.3068
03/19 22:49:39 - mmengine - INFO - Epoch(train)  [73][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:39:27  time: 0.2150  data_time: 0.0082  memory: 3357  loss: 4.3313  loss_cls: 0.6746  loss_bbox: 2.3527  loss_obj: 1.3039
03/19 22:49:49 - mmengine - INFO - Epoch(train)  [73][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:39:16  time: 0.2017  data_time: 0.0085  memory: 3357  loss: 4.2736  loss_cls: 0.6726  loss_bbox: 2.3191  loss_obj: 1.2818
03/19 22:50:02 - mmengine - INFO - Epoch(train)  [73][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:39:06  time: 0.2434  data_time: 0.0084  memory: 3937  loss: 4.2663  loss_cls: 0.6629  loss_bbox: 2.2995  loss_obj: 1.3038
03/19 22:50:12 - mmengine - INFO - Epoch(train)  [73][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:55  time: 0.2139  data_time: 0.0082  memory: 3639  loss: 4.3169  loss_cls: 0.6702  loss_bbox: 2.3322  loss_obj: 1.3146
03/19 22:50:22 - mmengine - INFO - Epoch(train)  [73][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:44  time: 0.1999  data_time: 0.0083  memory: 2817  loss: 4.2809  loss_cls: 0.6735  loss_bbox: 2.3295  loss_obj: 1.2779
03/19 22:50:33 - mmengine - INFO - Epoch(train)  [73][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:33  time: 0.2211  data_time: 0.0083  memory: 3937  loss: 4.3128  loss_cls: 0.6750  loss_bbox: 2.3295  loss_obj: 1.3084
03/19 22:50:45 - mmengine - INFO - Epoch(train)  [73][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:22  time: 0.2252  data_time: 0.0083  memory: 3937  loss: 4.2859  loss_cls: 0.6604  loss_bbox: 2.3274  loss_obj: 1.2981
03/19 22:50:56 - mmengine - INFO - Epoch(train)  [73][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:12  time: 0.2309  data_time: 0.0082  memory: 3639  loss: 4.3148  loss_cls: 0.6721  loss_bbox: 2.3239  loss_obj: 1.3189
03/19 22:51:08 - mmengine - INFO - Epoch(train)  [73][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:38:01  time: 0.2349  data_time: 0.0082  memory: 3639  loss: 4.2900  loss_cls: 0.6652  loss_bbox: 2.3148  loss_obj: 1.3100
03/19 22:51:18 - mmengine - INFO - Epoch(train)  [73][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:37:50  time: 0.1993  data_time: 0.0085  memory: 3937  loss: 4.3662  loss_cls: 0.6846  loss_bbox: 2.3569  loss_obj: 1.3246
03/19 22:51:29 - mmengine - INFO - Epoch(train)  [73][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:37:40  time: 0.2267  data_time: 0.0081  memory: 3071  loss: 4.3098  loss_cls: 0.6731  loss_bbox: 2.3250  loss_obj: 1.3117
03/19 22:51:40 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:51:40 - mmengine - INFO - Epoch(train)  [73][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:37:29  time: 0.2198  data_time: 0.0081  memory: 3937  loss: 4.3418  loss_cls: 0.6791  loss_bbox: 2.3653  loss_obj: 1.2975
03/19 22:51:40 - mmengine - INFO - Saving checkpoint at 73 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:51:46 - mmengine - INFO - Epoch(val)  [73][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0072  memory: 527  
03/19 22:51:49 - mmengine - INFO - Epoch(val)  [73][100/250]    eta: 0:00:08  time: 0.0579  data_time: 0.0065  memory: 527  
03/19 22:51:51 - mmengine - INFO - Epoch(val)  [73][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 22:51:54 - mmengine - INFO - Epoch(val)  [73][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 22:51:57 - mmengine - INFO - Epoch(val)  [73][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0065  memory: 527  
03/19 22:51:59 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.26s).
Accumulating evaluation results...
DONE (t=1.90s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.237
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.574
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.290
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.466
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.288
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.395
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.519
03/19 22:52:08 - mmengine - INFO - bbox_mAP_copypaste: 0.237 0.574 0.149 0.161 0.290 0.466
03/19 22:52:08 - mmengine - INFO - Epoch(val) [73][250/250]    coco/bbox_mAP: 0.2370  coco/bbox_mAP_50: 0.5740  coco/bbox_mAP_75: 0.1490  coco/bbox_mAP_s: 0.1610  coco/bbox_mAP_m: 0.2900  coco/bbox_mAP_l: 0.4660  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:52:20 - mmengine - INFO - Epoch(train)  [74][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:37:18  time: 0.2326  data_time: 0.0196  memory: 3357  loss: 4.2755  loss_cls: 0.6683  loss_bbox: 2.3303  loss_obj: 1.2769
03/19 22:52:31 - mmengine - INFO - Epoch(train)  [74][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:37:07  time: 0.2145  data_time: 0.0083  memory: 3357  loss: 4.3029  loss_cls: 0.6656  loss_bbox: 2.3387  loss_obj: 1.2987
03/19 22:52:41 - mmengine - INFO - Epoch(train)  [74][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:56  time: 0.2099  data_time: 0.0082  memory: 3357  loss: 4.2909  loss_cls: 0.6710  loss_bbox: 2.3189  loss_obj: 1.3009
03/19 22:52:53 - mmengine - INFO - Epoch(train)  [74][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:46  time: 0.2425  data_time: 0.0082  memory: 3937  loss: 4.3347  loss_cls: 0.6691  loss_bbox: 2.3212  loss_obj: 1.3445
03/19 22:53:05 - mmengine - INFO - Epoch(train)  [74][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:35  time: 0.2272  data_time: 0.0085  memory: 3639  loss: 4.2931  loss_cls: 0.6631  loss_bbox: 2.2976  loss_obj: 1.3323
03/19 22:53:16 - mmengine - INFO - Epoch(train)  [74][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:25  time: 0.2293  data_time: 0.0083  memory: 3639  loss: 4.3335  loss_cls: 0.6724  loss_bbox: 2.3231  loss_obj: 1.3379
03/19 22:53:27 - mmengine - INFO - Epoch(train)  [74][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:14  time: 0.2115  data_time: 0.0082  memory: 3639  loss: 4.2861  loss_cls: 0.6706  loss_bbox: 2.3081  loss_obj: 1.3074
03/19 22:53:37 - mmengine - INFO - Epoch(train)  [74][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:36:03  time: 0.2025  data_time: 0.0083  memory: 2587  loss: 4.2411  loss_cls: 0.6704  loss_bbox: 2.3206  loss_obj: 1.2501
03/19 22:53:47 - mmengine - INFO - Epoch(train)  [74][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:35:52  time: 0.2045  data_time: 0.0083  memory: 3357  loss: 4.2911  loss_cls: 0.6784  loss_bbox: 2.3380  loss_obj: 1.2747
03/19 22:53:59 - mmengine - INFO - Epoch(train)  [74][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:35:41  time: 0.2397  data_time: 0.0082  memory: 3937  loss: 4.3491  loss_cls: 0.6701  loss_bbox: 2.3258  loss_obj: 1.3532
03/19 22:54:09 - mmengine - INFO - Epoch(train)  [74][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:35:30  time: 0.1981  data_time: 0.0085  memory: 3071  loss: 4.3272  loss_cls: 0.6815  loss_bbox: 2.3412  loss_obj: 1.3044
03/19 22:54:20 - mmengine - INFO - Epoch(train)  [74][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:35:19  time: 0.2185  data_time: 0.0082  memory: 3639  loss: 4.3186  loss_cls: 0.6694  loss_bbox: 2.3406  loss_obj: 1.3085
03/19 22:54:31 - mmengine - INFO - Epoch(train)  [74][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:35:08  time: 0.2086  data_time: 0.0082  memory: 3071  loss: 4.2774  loss_cls: 0.6636  loss_bbox: 2.3489  loss_obj: 1.2649
03/19 22:54:43 - mmengine - INFO - Epoch(train)  [74][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:58  time: 0.2402  data_time: 0.0082  memory: 3639  loss: 4.2818  loss_cls: 0.6623  loss_bbox: 2.3247  loss_obj: 1.2947
03/19 22:54:54 - mmengine - INFO - Epoch(train)  [74][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:47  time: 0.2242  data_time: 0.0082  memory: 3357  loss: 4.3336  loss_cls: 0.6676  loss_bbox: 2.3303  loss_obj: 1.3357
03/19 22:55:05 - mmengine - INFO - Epoch(train)  [74][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:36  time: 0.2176  data_time: 0.0081  memory: 3639  loss: 4.3116  loss_cls: 0.6724  loss_bbox: 2.3464  loss_obj: 1.2928
03/19 22:55:16 - mmengine - INFO - Epoch(train)  [74][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:26  time: 0.2295  data_time: 0.0083  memory: 3639  loss: 4.3350  loss_cls: 0.6739  loss_bbox: 2.3356  loss_obj: 1.3255
03/19 22:55:27 - mmengine - INFO - Epoch(train)  [74][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:15  time: 0.2101  data_time: 0.0084  memory: 3357  loss: 4.3118  loss_cls: 0.6804  loss_bbox: 2.3351  loss_obj: 1.2963
03/19 22:55:37 - mmengine - INFO - Epoch(train)  [74][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:34:04  time: 0.1999  data_time: 0.0083  memory: 3639  loss: 4.3701  loss_cls: 0.6801  loss_bbox: 2.3449  loss_obj: 1.3451
03/19 22:55:47 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:55:47 - mmengine - INFO - Epoch(train)  [74][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:33:53  time: 0.2126  data_time: 0.0083  memory: 3639  loss: 4.3761  loss_cls: 0.6818  loss_bbox: 2.3646  loss_obj: 1.3297
03/19 22:55:47 - mmengine - INFO - Saving checkpoint at 74 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:55:53 - mmengine - INFO - Epoch(val)  [74][ 50/250]    eta: 0:00:11  time: 0.0597  data_time: 0.0072  memory: 527  
03/19 22:55:56 - mmengine - INFO - Epoch(val)  [74][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0065  memory: 527  
03/19 22:55:59 - mmengine - INFO - Epoch(val)  [74][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 22:56:02 - mmengine - INFO - Epoch(val)  [74][200/250]    eta: 0:00:02  time: 0.0587  data_time: 0.0066  memory: 527  
03/19 22:56:04 - mmengine - INFO - Epoch(val)  [74][250/250]    eta: 0:00:00  time: 0.0578  data_time: 0.0066  memory: 527  
03/19 22:56:06 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.38s).
Accumulating evaluation results...
DONE (t=1.91s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.238
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.576
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.149
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.292
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.461
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.287
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.516
03/19 22:56:16 - mmengine - INFO - bbox_mAP_copypaste: 0.238 0.576 0.149 0.161 0.292 0.461
03/19 22:56:16 - mmengine - INFO - Epoch(val) [74][250/250]    coco/bbox_mAP: 0.2380  coco/bbox_mAP_50: 0.5760  coco/bbox_mAP_75: 0.1490  coco/bbox_mAP_s: 0.1610  coco/bbox_mAP_m: 0.2920  coco/bbox_mAP_l: 0.4610  data_time: 0.0066  time: 0.0586
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 22:56:26 - mmengine - INFO - Epoch(train)  [75][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:33:42  time: 0.2099  data_time: 0.0185  memory: 3357  loss: 4.2513  loss_cls: 0.6613  loss_bbox: 2.3286  loss_obj: 1.2614
03/19 22:56:37 - mmengine - INFO - Epoch(train)  [75][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:33:31  time: 0.2227  data_time: 0.0082  memory: 3937  loss: 4.3215  loss_cls: 0.6629  loss_bbox: 2.3272  loss_obj: 1.3313
03/19 22:56:48 - mmengine - INFO - Epoch(train)  [75][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:33:20  time: 0.2065  data_time: 0.0084  memory: 3639  loss: 4.2879  loss_cls: 0.6682  loss_bbox: 2.3212  loss_obj: 1.2984
03/19 22:56:58 - mmengine - INFO - Epoch(train)  [75][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:33:09  time: 0.2117  data_time: 0.0083  memory: 3639  loss: 4.2883  loss_cls: 0.6664  loss_bbox: 2.3171  loss_obj: 1.3049
03/19 22:57:09 - mmengine - INFO - Epoch(train)  [75][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:58  time: 0.2111  data_time: 0.0082  memory: 3639  loss: 4.2624  loss_cls: 0.6685  loss_bbox: 2.3190  loss_obj: 1.2750
03/19 22:57:21 - mmengine - INFO - Epoch(train)  [75][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:48  time: 0.2397  data_time: 0.0082  memory: 3937  loss: 4.2762  loss_cls: 0.6645  loss_bbox: 2.3147  loss_obj: 1.2971
03/19 22:57:31 - mmengine - INFO - Epoch(train)  [75][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:37  time: 0.2096  data_time: 0.0083  memory: 3639  loss: 4.2337  loss_cls: 0.6644  loss_bbox: 2.3042  loss_obj: 1.2651
03/19 22:57:42 - mmengine - INFO - Epoch(train)  [75][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:26  time: 0.2061  data_time: 0.0084  memory: 3639  loss: 4.3404  loss_cls: 0.6800  loss_bbox: 2.3465  loss_obj: 1.3139
03/19 22:57:51 - mmengine - INFO - Epoch(train)  [75][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:15  time: 0.1930  data_time: 0.0085  memory: 3071  loss: 4.3453  loss_cls: 0.6839  loss_bbox: 2.3564  loss_obj: 1.3050
03/19 22:58:02 - mmengine - INFO - Epoch(train)  [75][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:32:04  time: 0.2041  data_time: 0.0086  memory: 3937  loss: 4.2844  loss_cls: 0.6734  loss_bbox: 2.3211  loss_obj: 1.2899
03/19 22:58:12 - mmengine - INFO - Epoch(train)  [75][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:31:53  time: 0.2145  data_time: 0.0081  memory: 3639  loss: 4.2902  loss_cls: 0.6638  loss_bbox: 2.3259  loss_obj: 1.3005
03/19 22:58:23 - mmengine - INFO - Epoch(train)  [75][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:31:42  time: 0.2166  data_time: 0.0082  memory: 3357  loss: 4.3215  loss_cls: 0.6717  loss_bbox: 2.3342  loss_obj: 1.3157
03/19 22:58:35 - mmengine - INFO - Epoch(train)  [75][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:31:31  time: 0.2258  data_time: 0.0082  memory: 3639  loss: 4.2930  loss_cls: 0.6760  loss_bbox: 2.3262  loss_obj: 1.2908
03/19 22:58:46 - mmengine - INFO - Epoch(train)  [75][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:31:21  time: 0.2205  data_time: 0.0083  memory: 3937  loss: 4.1956  loss_cls: 0.6676  loss_bbox: 2.2851  loss_obj: 1.2430
03/19 22:58:56 - mmengine - INFO - Epoch(train)  [75][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:31:10  time: 0.2167  data_time: 0.0082  memory: 3639  loss: 4.2939  loss_cls: 0.6691  loss_bbox: 2.3444  loss_obj: 1.2805
03/19 22:59:08 - mmengine - INFO - Epoch(train)  [75][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:59  time: 0.2371  data_time: 0.0082  memory: 3937  loss: 4.3226  loss_cls: 0.6711  loss_bbox: 2.3424  loss_obj: 1.3092
03/19 22:59:21 - mmengine - INFO - Epoch(train)  [75][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:49  time: 0.2443  data_time: 0.0082  memory: 3937  loss: 4.3645  loss_cls: 0.6792  loss_bbox: 2.3379  loss_obj: 1.3474
03/19 22:59:31 - mmengine - INFO - Epoch(train)  [75][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:38  time: 0.2143  data_time: 0.0084  memory: 3937  loss: 4.3487  loss_cls: 0.6770  loss_bbox: 2.3575  loss_obj: 1.3142
03/19 22:59:44 - mmengine - INFO - Epoch(train)  [75][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:28  time: 0.2531  data_time: 0.0083  memory: 3937  loss: 4.2671  loss_cls: 0.6628  loss_bbox: 2.2955  loss_obj: 1.3089
03/19 22:59:54 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 22:59:54 - mmengine - INFO - Epoch(train)  [75][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:17  time: 0.2093  data_time: 0.0081  memory: 3937  loss: 4.3008  loss_cls: 0.6710  loss_bbox: 2.3301  loss_obj: 1.2996
03/19 22:59:54 - mmengine - INFO - Saving checkpoint at 75 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:00:00 - mmengine - INFO - Epoch(val)  [75][ 50/250]    eta: 0:00:11  time: 0.0587  data_time: 0.0071  memory: 527  
03/19 23:00:03 - mmengine - INFO - Epoch(val)  [75][100/250]    eta: 0:00:08  time: 0.0591  data_time: 0.0066  memory: 527  
03/19 23:00:06 - mmengine - INFO - Epoch(val)  [75][150/250]    eta: 0:00:05  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 23:00:08 - mmengine - INFO - Epoch(val)  [75][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0064  memory: 527  
03/19 23:00:11 - mmengine - INFO - Epoch(val)  [75][250/250]    eta: 0:00:00  time: 0.0578  data_time: 0.0065  memory: 527  
03/19 23:00:13 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.27s).
Accumulating evaluation results...
DONE (t=1.90s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.238
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.162
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.293
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.467
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.287
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.396
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.517
03/19 23:00:22 - mmengine - INFO - bbox_mAP_copypaste: 0.238 0.577 0.150 0.162 0.293 0.467
03/19 23:00:22 - mmengine - INFO - Epoch(val) [75][250/250]    coco/bbox_mAP: 0.2380  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1620  coco/bbox_mAP_m: 0.2930  coco/bbox_mAP_l: 0.4670  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:00:34 - mmengine - INFO - Epoch(train)  [76][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:30:06  time: 0.2227  data_time: 0.0187  memory: 3071  loss: 4.2678  loss_cls: 0.6686  loss_bbox: 2.3377  loss_obj: 1.2615
03/19 23:00:44 - mmengine - INFO - Epoch(train)  [76][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:55  time: 0.2079  data_time: 0.0085  memory: 3639  loss: 4.3929  loss_cls: 0.6848  loss_bbox: 2.3663  loss_obj: 1.3418
03/19 23:00:55 - mmengine - INFO - Epoch(train)  [76][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:44  time: 0.2168  data_time: 0.0083  memory: 3639  loss: 4.2724  loss_cls: 0.6620  loss_bbox: 2.3268  loss_obj: 1.2836
03/19 23:01:07 - mmengine - INFO - Epoch(train)  [76][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:34  time: 0.2326  data_time: 0.0082  memory: 3639  loss: 4.3264  loss_cls: 0.6719  loss_bbox: 2.3293  loss_obj: 1.3253
03/19 23:01:18 - mmengine - INFO - Epoch(train)  [76][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:23  time: 0.2200  data_time: 0.0081  memory: 3357  loss: 4.2528  loss_cls: 0.6640  loss_bbox: 2.3241  loss_obj: 1.2647
03/19 23:01:28 - mmengine - INFO - Epoch(train)  [76][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:12  time: 0.2062  data_time: 0.0086  memory: 3357  loss: 4.2701  loss_cls: 0.6650  loss_bbox: 2.3069  loss_obj: 1.2983
03/19 23:01:38 - mmengine - INFO - Epoch(train)  [76][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:29:01  time: 0.2093  data_time: 0.0085  memory: 3357  loss: 4.3560  loss_cls: 0.6768  loss_bbox: 2.3619  loss_obj: 1.3173
03/19 23:01:49 - mmengine - INFO - Epoch(train)  [76][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:28:50  time: 0.2079  data_time: 0.0082  memory: 3639  loss: 4.3146  loss_cls: 0.6780  loss_bbox: 2.3401  loss_obj: 1.2965
03/19 23:02:00 - mmengine - INFO - Epoch(train)  [76][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:28:39  time: 0.2286  data_time: 0.0084  memory: 3937  loss: 4.2392  loss_cls: 0.6594  loss_bbox: 2.3106  loss_obj: 1.2692
03/19 23:02:10 - mmengine - INFO - Epoch(train)  [76][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:28:28  time: 0.2046  data_time: 0.0084  memory: 3937  loss: 4.3358  loss_cls: 0.6782  loss_bbox: 2.3554  loss_obj: 1.3021
03/19 23:02:21 - mmengine - INFO - Epoch(train)  [76][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:28:17  time: 0.2180  data_time: 0.0083  memory: 3071  loss: 4.3631  loss_cls: 0.6756  loss_bbox: 2.3508  loss_obj: 1.3367
03/19 23:02:32 - mmengine - INFO - Epoch(train)  [76][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:28:07  time: 0.2189  data_time: 0.0083  memory: 3639  loss: 4.2701  loss_cls: 0.6624  loss_bbox: 2.3069  loss_obj: 1.3007
03/19 23:02:44 - mmengine - INFO - Epoch(train)  [76][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:56  time: 0.2379  data_time: 0.0083  memory: 3937  loss: 4.3096  loss_cls: 0.6641  loss_bbox: 2.3216  loss_obj: 1.3239
03/19 23:02:55 - mmengine - INFO - Epoch(train)  [76][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:45  time: 0.2237  data_time: 0.0084  memory: 3937  loss: 4.3678  loss_cls: 0.6756  loss_bbox: 2.3344  loss_obj: 1.3578
03/19 23:03:07 - mmengine - INFO - Epoch(train)  [76][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:35  time: 0.2252  data_time: 0.0083  memory: 3937  loss: 4.2607  loss_cls: 0.6662  loss_bbox: 2.3178  loss_obj: 1.2767
03/19 23:03:18 - mmengine - INFO - Epoch(train)  [76][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:24  time: 0.2339  data_time: 0.0082  memory: 3937  loss: 4.3722  loss_cls: 0.6762  loss_bbox: 2.3464  loss_obj: 1.3497
03/19 23:03:29 - mmengine - INFO - Epoch(train)  [76][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:13  time: 0.2159  data_time: 0.0083  memory: 3639  loss: 4.2628  loss_cls: 0.6746  loss_bbox: 2.3165  loss_obj: 1.2716
03/19 23:03:40 - mmengine - INFO - Epoch(train)  [76][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:27:03  time: 0.2223  data_time: 0.0084  memory: 3937  loss: 4.4108  loss_cls: 0.6895  loss_bbox: 2.3539  loss_obj: 1.3674
03/19 23:03:51 - mmengine - INFO - Epoch(train)  [76][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:26:52  time: 0.2136  data_time: 0.0082  memory: 3357  loss: 4.3114  loss_cls: 0.6773  loss_bbox: 2.3196  loss_obj: 1.3145
03/19 23:04:02 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:04:02 - mmengine - INFO - Epoch(train)  [76][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:26:41  time: 0.2185  data_time: 0.0081  memory: 3071  loss: 4.3453  loss_cls: 0.6812  loss_bbox: 2.3371  loss_obj: 1.3270
03/19 23:04:02 - mmengine - INFO - Saving checkpoint at 76 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:04:07 - mmengine - INFO - Epoch(val)  [76][ 50/250]    eta: 0:00:11  time: 0.0599  data_time: 0.0073  memory: 527  
03/19 23:04:10 - mmengine - INFO - Epoch(val)  [76][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 23:04:13 - mmengine - INFO - Epoch(val)  [76][150/250]    eta: 0:00:05  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 23:04:16 - mmengine - INFO - Epoch(val)  [76][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:04:19 - mmengine - INFO - Epoch(val)  [76][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 23:04:20 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.28s).
Accumulating evaluation results...
DONE (t=1.88s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.238
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.574
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.294
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.483
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.344
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.344
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.344
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.284
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.398
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.531
03/19 23:04:30 - mmengine - INFO - bbox_mAP_copypaste: 0.238 0.574 0.150 0.161 0.294 0.483
03/19 23:04:30 - mmengine - INFO - Epoch(val) [76][250/250]    coco/bbox_mAP: 0.2380  coco/bbox_mAP_50: 0.5740  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1610  coco/bbox_mAP_m: 0.2940  coco/bbox_mAP_l: 0.4830  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:04:42 - mmengine - INFO - Epoch(train)  [77][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:26:30  time: 0.2289  data_time: 0.0184  memory: 3639  loss: 4.2760  loss_cls: 0.6778  loss_bbox: 2.3208  loss_obj: 1.2775
03/19 23:04:51 - mmengine - INFO - Epoch(train)  [77][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:26:19  time: 0.1965  data_time: 0.0083  memory: 2817  loss: 4.2722  loss_cls: 0.6681  loss_bbox: 2.3359  loss_obj: 1.2682
03/19 23:05:03 - mmengine - INFO - Epoch(train)  [77][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:26:09  time: 0.2347  data_time: 0.0083  memory: 3937  loss: 4.3028  loss_cls: 0.6674  loss_bbox: 2.3088  loss_obj: 1.3266
03/19 23:05:13 - mmengine - INFO - Epoch(train)  [77][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:58  time: 0.2055  data_time: 0.0083  memory: 3071  loss: 4.2872  loss_cls: 0.6720  loss_bbox: 2.3254  loss_obj: 1.2898
03/19 23:05:25 - mmengine - INFO - Epoch(train)  [77][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:47  time: 0.2299  data_time: 0.0083  memory: 3937  loss: 4.2682  loss_cls: 0.6606  loss_bbox: 2.3108  loss_obj: 1.2968
03/19 23:05:36 - mmengine - INFO - Epoch(train)  [77][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:36  time: 0.2285  data_time: 0.0083  memory: 3937  loss: 4.2993  loss_cls: 0.6620  loss_bbox: 2.3229  loss_obj: 1.3144
03/19 23:05:47 - mmengine - INFO - Epoch(train)  [77][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:25  time: 0.2166  data_time: 0.0082  memory: 3937  loss: 4.3745  loss_cls: 0.6738  loss_bbox: 2.3602  loss_obj: 1.3405
03/19 23:05:58 - mmengine - INFO - Epoch(train)  [77][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:14  time: 0.2063  data_time: 0.0083  memory: 3357  loss: 4.2952  loss_cls: 0.6752  loss_bbox: 2.3279  loss_obj: 1.2921
03/19 23:06:08 - mmengine - INFO - Epoch(train)  [77][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:25:04  time: 0.2136  data_time: 0.0083  memory: 3639  loss: 4.2583  loss_cls: 0.6713  loss_bbox: 2.3123  loss_obj: 1.2747
03/19 23:06:18 - mmengine - INFO - Epoch(train)  [77][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:24:52  time: 0.2014  data_time: 0.0085  memory: 3639  loss: 4.3555  loss_cls: 0.6860  loss_bbox: 2.3643  loss_obj: 1.3052
03/19 23:06:29 - mmengine - INFO - Epoch(train)  [77][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:24:42  time: 0.2179  data_time: 0.0082  memory: 3071  loss: 4.2904  loss_cls: 0.6639  loss_bbox: 2.3160  loss_obj: 1.3104
03/19 23:06:40 - mmengine - INFO - Epoch(train)  [77][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:24:31  time: 0.2130  data_time: 0.0083  memory: 3357  loss: 4.2550  loss_cls: 0.6651  loss_bbox: 2.3153  loss_obj: 1.2746
03/19 23:06:51 - mmengine - INFO - Epoch(train)  [77][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:24:20  time: 0.2269  data_time: 0.0082  memory: 3639  loss: 4.3139  loss_cls: 0.6724  loss_bbox: 2.3413  loss_obj: 1.3002
03/19 23:07:01 - mmengine - INFO - Epoch(train)  [77][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:24:09  time: 0.1901  data_time: 0.0085  memory: 3357  loss: 4.3099  loss_cls: 0.6718  loss_bbox: 2.3350  loss_obj: 1.3031
03/19 23:07:13 - mmengine - INFO - Epoch(train)  [77][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:58  time: 0.2372  data_time: 0.0082  memory: 3937  loss: 4.2805  loss_cls: 0.6611  loss_bbox: 2.3258  loss_obj: 1.2936
03/19 23:07:23 - mmengine - INFO - Epoch(train)  [77][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:47  time: 0.2045  data_time: 0.0082  memory: 3071  loss: 4.2452  loss_cls: 0.6675  loss_bbox: 2.3121  loss_obj: 1.2656
03/19 23:07:34 - mmengine - INFO - Epoch(train)  [77][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:37  time: 0.2237  data_time: 0.0083  memory: 3639  loss: 4.2527  loss_cls: 0.6586  loss_bbox: 2.3080  loss_obj: 1.2861
03/19 23:07:46 - mmengine - INFO - Epoch(train)  [77][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:26  time: 0.2379  data_time: 0.0083  memory: 3937  loss: 4.2861  loss_cls: 0.6675  loss_bbox: 2.3062  loss_obj: 1.3124
03/19 23:07:57 - mmengine - INFO - Epoch(train)  [77][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:15  time: 0.2206  data_time: 0.0083  memory: 3639  loss: 4.2722  loss_cls: 0.6684  loss_bbox: 2.3078  loss_obj: 1.2959
03/19 23:08:07 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:08:07 - mmengine - INFO - Epoch(train)  [77][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:23:04  time: 0.2035  data_time: 0.0083  memory: 3937  loss: 4.3088  loss_cls: 0.6758  loss_bbox: 2.3372  loss_obj: 1.2958
03/19 23:08:07 - mmengine - INFO - Saving checkpoint at 77 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:08:13 - mmengine - INFO - Epoch(val)  [77][ 50/250]    eta: 0:00:11  time: 0.0586  data_time: 0.0072  memory: 527  
03/19 23:08:16 - mmengine - INFO - Epoch(val)  [77][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:08:18 - mmengine - INFO - Epoch(val)  [77][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:08:21 - mmengine - INFO - Epoch(val)  [77][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0064  memory: 527  
03/19 23:08:24 - mmengine - INFO - Epoch(val)  [77][250/250]    eta: 0:00:00  time: 0.0576  data_time: 0.0064  memory: 527  
03/19 23:08:26 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.22s).
Accumulating evaluation results...
DONE (t=1.87s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.240
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.164
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.298
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.485
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.290
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.398
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.535
03/19 23:08:35 - mmengine - INFO - bbox_mAP_copypaste: 0.240 0.577 0.151 0.164 0.298 0.485
03/19 23:08:35 - mmengine - INFO - Epoch(val) [77][250/250]    coco/bbox_mAP: 0.2400  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1640  coco/bbox_mAP_m: 0.2980  coco/bbox_mAP_l: 0.4850  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:08:46 - mmengine - INFO - Epoch(train)  [78][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:53  time: 0.2068  data_time: 0.0195  memory: 2587  loss: 4.3594  loss_cls: 0.6771  loss_bbox: 2.3605  loss_obj: 1.3217
03/19 23:08:58 - mmengine - INFO - Epoch(train)  [78][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:43  time: 0.2411  data_time: 0.0083  memory: 3937  loss: 4.2822  loss_cls: 0.6735  loss_bbox: 2.2940  loss_obj: 1.3148
03/19 23:09:10 - mmengine - INFO - Epoch(train)  [78][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:32  time: 0.2392  data_time: 0.0082  memory: 3937  loss: 4.2937  loss_cls: 0.6680  loss_bbox: 2.3126  loss_obj: 1.3130
03/19 23:09:22 - mmengine - INFO - Epoch(train)  [78][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:22  time: 0.2377  data_time: 0.0083  memory: 3937  loss: 4.3153  loss_cls: 0.6688  loss_bbox: 2.3144  loss_obj: 1.3320
03/19 23:09:33 - mmengine - INFO - Epoch(train)  [78][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:11  time: 0.2184  data_time: 0.0083  memory: 3937  loss: 4.2328  loss_cls: 0.6606  loss_bbox: 2.2822  loss_obj: 1.2900
03/19 23:09:43 - mmengine - INFO - Epoch(train)  [78][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:22:00  time: 0.2071  data_time: 0.0083  memory: 3071  loss: 4.2793  loss_cls: 0.6724  loss_bbox: 2.3292  loss_obj: 1.2778
03/19 23:09:53 - mmengine - INFO - Epoch(train)  [78][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:21:49  time: 0.2065  data_time: 0.0082  memory: 2817  loss: 4.2009  loss_cls: 0.6603  loss_bbox: 2.3015  loss_obj: 1.2392
03/19 23:10:04 - mmengine - INFO - Epoch(train)  [78][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:21:38  time: 0.2208  data_time: 0.0082  memory: 3071  loss: 4.3084  loss_cls: 0.6701  loss_bbox: 2.3360  loss_obj: 1.3024
03/19 23:10:14 - mmengine - INFO - Epoch(train)  [78][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:21:27  time: 0.2017  data_time: 0.0086  memory: 3639  loss: 4.2771  loss_cls: 0.6698  loss_bbox: 2.3107  loss_obj: 1.2966
03/19 23:10:25 - mmengine - INFO - Epoch(train)  [78][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:21:16  time: 0.2058  data_time: 0.0082  memory: 3071  loss: 4.2254  loss_cls: 0.6657  loss_bbox: 2.2951  loss_obj: 1.2646
03/19 23:10:36 - mmengine - INFO - Epoch(train)  [78][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:21:06  time: 0.2343  data_time: 0.0083  memory: 3937  loss: 4.3336  loss_cls: 0.6778  loss_bbox: 2.3285  loss_obj: 1.3274
03/19 23:10:49 - mmengine - INFO - Epoch(train)  [78][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:55  time: 0.2487  data_time: 0.0082  memory: 3937  loss: 4.2943  loss_cls: 0.6620  loss_bbox: 2.2976  loss_obj: 1.3347
03/19 23:11:01 - mmengine - INFO - Epoch(train)  [78][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:45  time: 0.2441  data_time: 0.0082  memory: 3937  loss: 4.2973  loss_cls: 0.6662  loss_bbox: 2.3402  loss_obj: 1.2909
03/19 23:11:12 - mmengine - INFO - Epoch(train)  [78][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:34  time: 0.2182  data_time: 0.0083  memory: 3639  loss: 4.2394  loss_cls: 0.6517  loss_bbox: 2.3077  loss_obj: 1.2800
03/19 23:11:23 - mmengine - INFO - Epoch(train)  [78][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:23  time: 0.2210  data_time: 0.0082  memory: 3937  loss: 4.2189  loss_cls: 0.6590  loss_bbox: 2.3064  loss_obj: 1.2535
03/19 23:11:33 - mmengine - INFO - Epoch(train)  [78][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:12  time: 0.1993  data_time: 0.0084  memory: 3639  loss: 4.3446  loss_cls: 0.6835  loss_bbox: 2.3455  loss_obj: 1.3156
03/19 23:11:44 - mmengine - INFO - Epoch(train)  [78][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:20:01  time: 0.2174  data_time: 0.0083  memory: 3639  loss: 4.2846  loss_cls: 0.6715  loss_bbox: 2.3334  loss_obj: 1.2798
03/19 23:11:55 - mmengine - INFO - Epoch(train)  [78][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:19:50  time: 0.2189  data_time: 0.0083  memory: 3639  loss: 4.2905  loss_cls: 0.6631  loss_bbox: 2.3377  loss_obj: 1.2897
03/19 23:12:06 - mmengine - INFO - Epoch(train)  [78][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:19:40  time: 0.2186  data_time: 0.0085  memory: 3937  loss: 4.2620  loss_cls: 0.6641  loss_bbox: 2.3387  loss_obj: 1.2592
03/19 23:12:17 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:12:17 - mmengine - INFO - Epoch(train)  [78][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:19:29  time: 0.2178  data_time: 0.0081  memory: 3639  loss: 4.2533  loss_cls: 0.6715  loss_bbox: 2.3046  loss_obj: 1.2773
03/19 23:12:17 - mmengine - INFO - Saving checkpoint at 78 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:12:22 - mmengine - INFO - Epoch(val)  [78][ 50/250]    eta: 0:00:11  time: 0.0596  data_time: 0.0072  memory: 527  
03/19 23:12:25 - mmengine - INFO - Epoch(val)  [78][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0064  memory: 527  
03/19 23:12:28 - mmengine - INFO - Epoch(val)  [78][150/250]    eta: 0:00:05  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 23:12:31 - mmengine - INFO - Epoch(val)  [78][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:12:34 - mmengine - INFO - Epoch(val)  [78][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0065  memory: 527  
03/19 23:12:35 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.00s).
Accumulating evaluation results...
DONE (t=2.09s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.575
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.164
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.296
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.483
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.349
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.291
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.399
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.531
03/19 23:12:45 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.575 0.151 0.164 0.296 0.483
03/19 23:12:45 - mmengine - INFO - Epoch(val) [78][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5750  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1640  coco/bbox_mAP_m: 0.2960  coco/bbox_mAP_l: 0.4830  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:12:56 - mmengine - INFO - Epoch(train)  [79][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:19:18  time: 0.2298  data_time: 0.0186  memory: 3639  loss: 4.2097  loss_cls: 0.6548  loss_bbox: 2.3018  loss_obj: 1.2531
03/19 23:13:09 - mmengine - INFO - Epoch(train)  [79][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:19:08  time: 0.2506  data_time: 0.0083  memory: 3937  loss: 4.2743  loss_cls: 0.6603  loss_bbox: 2.3145  loss_obj: 1.2994
03/19 23:13:18 - mmengine - INFO - Epoch(train)  [79][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:56  time: 0.1885  data_time: 0.0084  memory: 2337  loss: 4.2938  loss_cls: 0.6724  loss_bbox: 2.3554  loss_obj: 1.2659
03/19 23:13:30 - mmengine - INFO - Epoch(train)  [79][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:46  time: 0.2317  data_time: 0.0083  memory: 3937  loss: 4.2811  loss_cls: 0.6716  loss_bbox: 2.3088  loss_obj: 1.3007
03/19 23:13:41 - mmengine - INFO - Epoch(train)  [79][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:35  time: 0.2148  data_time: 0.0083  memory: 3937  loss: 4.3508  loss_cls: 0.6750  loss_bbox: 2.3498  loss_obj: 1.3261
03/19 23:13:51 - mmengine - INFO - Epoch(train)  [79][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:24  time: 0.2112  data_time: 0.0083  memory: 3937  loss: 4.2808  loss_cls: 0.6713  loss_bbox: 2.3127  loss_obj: 1.2968
03/19 23:14:03 - mmengine - INFO - Epoch(train)  [79][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:13  time: 0.2346  data_time: 0.0084  memory: 3937  loss: 4.2754  loss_cls: 0.6669  loss_bbox: 2.3288  loss_obj: 1.2797
03/19 23:14:14 - mmengine - INFO - Epoch(train)  [79][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:18:03  time: 0.2259  data_time: 0.0083  memory: 3639  loss: 4.2888  loss_cls: 0.6689  loss_bbox: 2.3168  loss_obj: 1.3031
03/19 23:14:26 - mmengine - INFO - Epoch(train)  [79][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:17:52  time: 0.2408  data_time: 0.0082  memory: 3639  loss: 4.2594  loss_cls: 0.6667  loss_bbox: 2.2989  loss_obj: 1.2937
03/19 23:14:37 - mmengine - INFO - Epoch(train)  [79][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:17:41  time: 0.2221  data_time: 0.0083  memory: 3357  loss: 4.3571  loss_cls: 0.6822  loss_bbox: 2.3479  loss_obj: 1.3269
03/19 23:14:48 - mmengine - INFO - Epoch(train)  [79][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:17:31  time: 0.2115  data_time: 0.0081  memory: 3071  loss: 4.2498  loss_cls: 0.6702  loss_bbox: 2.3081  loss_obj: 1.2714
03/19 23:14:58 - mmengine - INFO - Epoch(train)  [79][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:17:19  time: 0.1924  data_time: 0.0085  memory: 3071  loss: 4.3114  loss_cls: 0.6724  loss_bbox: 2.3354  loss_obj: 1.3035
03/19 23:15:09 - mmengine - INFO - Epoch(train)  [79][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:17:09  time: 0.2309  data_time: 0.0083  memory: 3937  loss: 4.2691  loss_cls: 0.6638  loss_bbox: 2.3085  loss_obj: 1.2968
03/19 23:15:21 - mmengine - INFO - Epoch(train)  [79][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:58  time: 0.2324  data_time: 0.0082  memory: 3639  loss: 4.3462  loss_cls: 0.6681  loss_bbox: 2.3277  loss_obj: 1.3504
03/19 23:15:33 - mmengine - INFO - Epoch(train)  [79][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:48  time: 0.2377  data_time: 0.0083  memory: 3639  loss: 4.3418  loss_cls: 0.6722  loss_bbox: 2.3258  loss_obj: 1.3438
03/19 23:15:44 - mmengine - INFO - Epoch(train)  [79][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:37  time: 0.2223  data_time: 0.0082  memory: 3937  loss: 4.2872  loss_cls: 0.6651  loss_bbox: 2.3222  loss_obj: 1.2999
03/19 23:15:55 - mmengine - INFO - Epoch(train)  [79][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:26  time: 0.2128  data_time: 0.0084  memory: 3937  loss: 4.3726  loss_cls: 0.6782  loss_bbox: 2.3705  loss_obj: 1.3239
03/19 23:16:05 - mmengine - INFO - Epoch(train)  [79][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:15  time: 0.2148  data_time: 0.0084  memory: 3357  loss: 4.3016  loss_cls: 0.6695  loss_bbox: 2.3371  loss_obj: 1.2951
03/19 23:16:15 - mmengine - INFO - Epoch(train)  [79][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:16:04  time: 0.2021  data_time: 0.0085  memory: 3357  loss: 4.2498  loss_cls: 0.6810  loss_bbox: 2.3181  loss_obj: 1.2506
03/19 23:16:26 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:16:26 - mmengine - INFO - Epoch(train)  [79][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:15:53  time: 0.2015  data_time: 0.0081  memory: 3071  loss: 4.2512  loss_cls: 0.6750  loss_bbox: 2.3126  loss_obj: 1.2636
03/19 23:16:26 - mmengine - INFO - Saving checkpoint at 79 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:16:31 - mmengine - INFO - Epoch(val)  [79][ 50/250]    eta: 0:00:11  time: 0.0598  data_time: 0.0071  memory: 527  
03/19 23:16:34 - mmengine - INFO - Epoch(val)  [79][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:16:37 - mmengine - INFO - Epoch(val)  [79][150/250]    eta: 0:00:05  time: 0.0582  data_time: 0.0065  memory: 527  
03/19 23:16:40 - mmengine - INFO - Epoch(val)  [79][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 23:16:43 - mmengine - INFO - Epoch(val)  [79][250/250]    eta: 0:00:00  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 23:16:44 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.26s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=6.96s).
Accumulating evaluation results...
DONE (t=1.85s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.153
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.164
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.297
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.485
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.288
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.398
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.534
03/19 23:16:53 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.577 0.153 0.164 0.297 0.485
03/19 23:16:53 - mmengine - INFO - Epoch(val) [79][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1530  coco/bbox_mAP_s: 0.1640  coco/bbox_mAP_m: 0.2970  coco/bbox_mAP_l: 0.4850  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:17:06 - mmengine - INFO - Epoch(train)  [80][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:15:42  time: 0.2461  data_time: 0.0189  memory: 3639  loss: 4.2136  loss_cls: 0.6616  loss_bbox: 2.3012  loss_obj: 1.2508
03/19 23:17:16 - mmengine - INFO - Epoch(train)  [80][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:15:31  time: 0.2014  data_time: 0.0083  memory: 3071  loss: 4.2914  loss_cls: 0.6728  loss_bbox: 2.3497  loss_obj: 1.2689
03/19 23:17:27 - mmengine - INFO - Epoch(train)  [80][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:15:21  time: 0.2203  data_time: 0.0082  memory: 3937  loss: 4.2526  loss_cls: 0.6685  loss_bbox: 2.3138  loss_obj: 1.2703
03/19 23:17:37 - mmengine - INFO - Epoch(train)  [80][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:15:10  time: 0.2152  data_time: 0.0083  memory: 3639  loss: 4.3508  loss_cls: 0.6753  loss_bbox: 2.3544  loss_obj: 1.3211
03/19 23:17:49 - mmengine - INFO - Epoch(train)  [80][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:59  time: 0.2355  data_time: 0.0084  memory: 3937  loss: 4.1889  loss_cls: 0.6509  loss_bbox: 2.2820  loss_obj: 1.2560
03/19 23:18:00 - mmengine - INFO - Epoch(train)  [80][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:48  time: 0.2114  data_time: 0.0083  memory: 2817  loss: 4.3033  loss_cls: 0.6811  loss_bbox: 2.3310  loss_obj: 1.2912
03/19 23:18:11 - mmengine - INFO - Epoch(train)  [80][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:38  time: 0.2287  data_time: 0.0084  memory: 3937  loss: 4.2355  loss_cls: 0.6653  loss_bbox: 2.3020  loss_obj: 1.2681
03/19 23:18:23 - mmengine - INFO - Epoch(train)  [80][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:27  time: 0.2343  data_time: 0.0082  memory: 3937  loss: 4.2401  loss_cls: 0.6543  loss_bbox: 2.3040  loss_obj: 1.2818
03/19 23:18:34 - mmengine - INFO - Epoch(train)  [80][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:16  time: 0.2134  data_time: 0.0082  memory: 3639  loss: 4.2636  loss_cls: 0.6607  loss_bbox: 2.3015  loss_obj: 1.3015
03/19 23:18:44 - mmengine - INFO - Epoch(train)  [80][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:14:05  time: 0.2048  data_time: 0.0083  memory: 3357  loss: 4.2679  loss_cls: 0.6663  loss_bbox: 2.3187  loss_obj: 1.2829
03/19 23:18:55 - mmengine - INFO - Epoch(train)  [80][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:54  time: 0.2210  data_time: 0.0084  memory: 3937  loss: 4.3021  loss_cls: 0.6748  loss_bbox: 2.3231  loss_obj: 1.3042
03/19 23:19:05 - mmengine - INFO - Epoch(train)  [80][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:43  time: 0.1997  data_time: 0.0083  memory: 2817  loss: 4.3140  loss_cls: 0.6767  loss_bbox: 2.3351  loss_obj: 1.3022
03/19 23:19:15 - mmengine - INFO - Epoch(train)  [80][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:32  time: 0.1930  data_time: 0.0083  memory: 2337  loss: 4.2548  loss_cls: 0.6652  loss_bbox: 2.3289  loss_obj: 1.2607
03/19 23:19:26 - mmengine - INFO - Epoch(train)  [80][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:21  time: 0.2195  data_time: 0.0084  memory: 3937  loss: 4.3369  loss_cls: 0.6713  loss_bbox: 2.3422  loss_obj: 1.3234
03/19 23:19:36 - mmengine - INFO - Epoch(train)  [80][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:10  time: 0.2052  data_time: 0.0082  memory: 2817  loss: 4.2932  loss_cls: 0.6701  loss_bbox: 2.3390  loss_obj: 1.2841
03/19 23:19:48 - mmengine - INFO - Epoch(train)  [80][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:13:00  time: 0.2357  data_time: 0.0083  memory: 3937  loss: 4.2463  loss_cls: 0.6596  loss_bbox: 2.2906  loss_obj: 1.2961
03/19 23:20:00 - mmengine - INFO - Epoch(train)  [80][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:12:49  time: 0.2349  data_time: 0.0083  memory: 3937  loss: 4.3178  loss_cls: 0.6760  loss_bbox: 2.3252  loss_obj: 1.3166
03/19 23:20:11 - mmengine - INFO - Epoch(train)  [80][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:12:38  time: 0.2349  data_time: 0.0084  memory: 3639  loss: 4.2880  loss_cls: 0.6675  loss_bbox: 2.3073  loss_obj: 1.3132
03/19 23:20:22 - mmengine - INFO - Epoch(train)  [80][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:12:28  time: 0.2161  data_time: 0.0084  memory: 3937  loss: 4.2796  loss_cls: 0.6717  loss_bbox: 2.3319  loss_obj: 1.2761
03/19 23:20:33 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:20:33 - mmengine - INFO - Epoch(train)  [80][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:12:17  time: 0.2131  data_time: 0.0081  memory: 3639  loss: 4.2712  loss_cls: 0.6629  loss_bbox: 2.3270  loss_obj: 1.2813
03/19 23:20:33 - mmengine - INFO - Saving checkpoint at 80 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:20:38 - mmengine - INFO - Epoch(val)  [80][ 50/250]    eta: 0:00:11  time: 0.0585  data_time: 0.0071  memory: 527  
03/19 23:20:41 - mmengine - INFO - Epoch(val)  [80][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 23:20:44 - mmengine - INFO - Epoch(val)  [80][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:20:47 - mmengine - INFO - Epoch(val)  [80][200/250]    eta: 0:00:02  time: 0.0594  data_time: 0.0065  memory: 527  
03/19 23:20:50 - mmengine - INFO - Epoch(val)  [80][250/250]    eta: 0:00:00  time: 0.0580  data_time: 0.0065  memory: 527  
03/19 23:20:51 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.23s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.18s).
Accumulating evaluation results...
DONE (t=1.86s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.163
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.294
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.486
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.346
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.289
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.385
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.537
03/19 23:21:01 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.577 0.151 0.163 0.294 0.486
03/19 23:21:01 - mmengine - INFO - Epoch(val) [80][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1630  coco/bbox_mAP_m: 0.2940  coco/bbox_mAP_l: 0.4860  data_time: 0.0066  time: 0.0585
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:21:11 - mmengine - INFO - Epoch(train)  [81][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:12:06  time: 0.2114  data_time: 0.0190  memory: 2817  loss: 4.2874  loss_cls: 0.6681  loss_bbox: 2.3128  loss_obj: 1.3064
03/19 23:21:22 - mmengine - INFO - Epoch(train)  [81][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:55  time: 0.2198  data_time: 0.0081  memory: 3639  loss: 4.2591  loss_cls: 0.6696  loss_bbox: 2.2887  loss_obj: 1.3007
03/19 23:21:33 - mmengine - INFO - Epoch(train)  [81][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:44  time: 0.2212  data_time: 0.0084  memory: 3937  loss: 4.3483  loss_cls: 0.6809  loss_bbox: 2.3355  loss_obj: 1.3319
03/19 23:21:43 - mmengine - INFO - Epoch(train)  [81][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:33  time: 0.1899  data_time: 0.0086  memory: 2817  loss: 4.3224  loss_cls: 0.6782  loss_bbox: 2.3492  loss_obj: 1.2950
03/19 23:21:52 - mmengine - INFO - Epoch(train)  [81][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:22  time: 0.1898  data_time: 0.0085  memory: 3071  loss: 4.2950  loss_cls: 0.6791  loss_bbox: 2.3405  loss_obj: 1.2755
03/19 23:22:04 - mmengine - INFO - Epoch(train)  [81][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:11  time: 0.2237  data_time: 0.0082  memory: 3639  loss: 4.3058  loss_cls: 0.6702  loss_bbox: 2.3191  loss_obj: 1.3165
03/19 23:22:14 - mmengine - INFO - Epoch(train)  [81][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:11:00  time: 0.2072  data_time: 0.0083  memory: 3357  loss: 4.3441  loss_cls: 0.6772  loss_bbox: 2.3407  loss_obj: 1.3262
03/19 23:22:24 - mmengine - INFO - Epoch(train)  [81][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:10:49  time: 0.2006  data_time: 0.0083  memory: 2587  loss: 4.2892  loss_cls: 0.6708  loss_bbox: 2.3475  loss_obj: 1.2709
03/19 23:22:35 - mmengine - INFO - Epoch(train)  [81][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:10:38  time: 0.2102  data_time: 0.0082  memory: 3071  loss: 4.3255  loss_cls: 0.6751  loss_bbox: 2.3341  loss_obj: 1.3163
03/19 23:22:46 - mmengine - INFO - Epoch(train)  [81][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:10:28  time: 0.2367  data_time: 0.0083  memory: 3937  loss: 4.2839  loss_cls: 0.6691  loss_bbox: 2.3149  loss_obj: 1.2999
03/19 23:22:57 - mmengine - INFO - Epoch(train)  [81][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:10:17  time: 0.2082  data_time: 0.0083  memory: 3071  loss: 4.2451  loss_cls: 0.6601  loss_bbox: 2.3226  loss_obj: 1.2624
03/19 23:23:09 - mmengine - INFO - Epoch(train)  [81][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:10:06  time: 0.2366  data_time: 0.0082  memory: 3937  loss: 4.2449  loss_cls: 0.6578  loss_bbox: 2.3131  loss_obj: 1.2740
03/19 23:23:20 - mmengine - INFO - Epoch(train)  [81][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:55  time: 0.2204  data_time: 0.0083  memory: 3639  loss: 4.3064  loss_cls: 0.6594  loss_bbox: 2.3213  loss_obj: 1.3258
03/19 23:23:31 - mmengine - INFO - Epoch(train)  [81][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:44  time: 0.2231  data_time: 0.0083  memory: 3937  loss: 4.3202  loss_cls: 0.6730  loss_bbox: 2.3365  loss_obj: 1.3107
03/19 23:23:43 - mmengine - INFO - Epoch(train)  [81][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:34  time: 0.2321  data_time: 0.0083  memory: 3937  loss: 4.3433  loss_cls: 0.6677  loss_bbox: 2.3559  loss_obj: 1.3196
03/19 23:23:54 - mmengine - INFO - Epoch(train)  [81][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:23  time: 0.2224  data_time: 0.0083  memory: 3937  loss: 4.3428  loss_cls: 0.6727  loss_bbox: 2.3551  loss_obj: 1.3150
03/19 23:24:04 - mmengine - INFO - Epoch(train)  [81][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:12  time: 0.2146  data_time: 0.0084  memory: 3937  loss: 4.2885  loss_cls: 0.6666  loss_bbox: 2.3310  loss_obj: 1.2909
03/19 23:24:15 - mmengine - INFO - Epoch(train)  [81][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:09:01  time: 0.2026  data_time: 0.0085  memory: 3357  loss: 4.2783  loss_cls: 0.6734  loss_bbox: 2.3393  loss_obj: 1.2656
03/19 23:24:26 - mmengine - INFO - Epoch(train)  [81][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:08:50  time: 0.2352  data_time: 0.0082  memory: 3357  loss: 4.2665  loss_cls: 0.6635  loss_bbox: 2.3071  loss_obj: 1.2958
03/19 23:24:37 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:24:37 - mmengine - INFO - Epoch(train)  [81][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:08:40  time: 0.2221  data_time: 0.0081  memory: 3639  loss: 4.2914  loss_cls: 0.6715  loss_bbox: 2.3217  loss_obj: 1.2982
03/19 23:24:37 - mmengine - INFO - Saving checkpoint at 81 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:24:43 - mmengine - INFO - Epoch(val)  [81][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0071  memory: 527  
03/19 23:24:46 - mmengine - INFO - Epoch(val)  [81][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0064  memory: 527  
03/19 23:24:49 - mmengine - INFO - Epoch(val)  [81][150/250]    eta: 0:00:05  time: 0.0582  data_time: 0.0064  memory: 527  
03/19 23:24:52 - mmengine - INFO - Epoch(val)  [81][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:24:54 - mmengine - INFO - Epoch(val)  [81][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0065  memory: 527  
03/19 23:24:56 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.21s).
Accumulating evaluation results...
DONE (t=1.86s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.579
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.163
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.296
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.489
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.289
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.537
03/19 23:25:05 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.579 0.151 0.163 0.296 0.489
03/19 23:25:05 - mmengine - INFO - Epoch(val) [81][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5790  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1630  coco/bbox_mAP_m: 0.2960  coco/bbox_mAP_l: 0.4890  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:25:17 - mmengine - INFO - Epoch(train)  [82][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:08:29  time: 0.2262  data_time: 0.0190  memory: 3357  loss: 4.3092  loss_cls: 0.6726  loss_bbox: 2.3475  loss_obj: 1.2890
03/19 23:25:28 - mmengine - INFO - Epoch(train)  [82][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:08:18  time: 0.2197  data_time: 0.0083  memory: 3937  loss: 4.2806  loss_cls: 0.6699  loss_bbox: 2.3354  loss_obj: 1.2753
03/19 23:25:39 - mmengine - INFO - Epoch(train)  [82][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:08:07  time: 0.2279  data_time: 0.0082  memory: 3937  loss: 4.2789  loss_cls: 0.6563  loss_bbox: 2.3256  loss_obj: 1.2969
03/19 23:25:49 - mmengine - INFO - Epoch(train)  [82][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:56  time: 0.1986  data_time: 0.0084  memory: 2817  loss: 4.2099  loss_cls: 0.6603  loss_bbox: 2.3033  loss_obj: 1.2463
03/19 23:26:00 - mmengine - INFO - Epoch(train)  [82][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:46  time: 0.2170  data_time: 0.0083  memory: 3937  loss: 4.2605  loss_cls: 0.6596  loss_bbox: 2.3089  loss_obj: 1.2920
03/19 23:26:10 - mmengine - INFO - Epoch(train)  [82][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:35  time: 0.2019  data_time: 0.0083  memory: 2817  loss: 4.2567  loss_cls: 0.6689  loss_bbox: 2.3190  loss_obj: 1.2688
03/19 23:26:20 - mmengine - INFO - Epoch(train)  [82][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:24  time: 0.2029  data_time: 0.0084  memory: 3937  loss: 4.3778  loss_cls: 0.6790  loss_bbox: 2.3658  loss_obj: 1.3330
03/19 23:26:30 - mmengine - INFO - Epoch(train)  [82][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:13  time: 0.2031  data_time: 0.0083  memory: 3071  loss: 4.2951  loss_cls: 0.6780  loss_bbox: 2.3414  loss_obj: 1.2757
03/19 23:26:42 - mmengine - INFO - Epoch(train)  [82][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:07:02  time: 0.2294  data_time: 0.0083  memory: 3639  loss: 4.2950  loss_cls: 0.6698  loss_bbox: 2.3222  loss_obj: 1.3031
03/19 23:26:53 - mmengine - INFO - Epoch(train)  [82][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:06:51  time: 0.2216  data_time: 0.0084  memory: 3937  loss: 4.3587  loss_cls: 0.6813  loss_bbox: 2.3462  loss_obj: 1.3312
03/19 23:27:04 - mmengine - INFO - Epoch(train)  [82][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:06:40  time: 0.2116  data_time: 0.0084  memory: 3357  loss: 4.3192  loss_cls: 0.6717  loss_bbox: 2.3398  loss_obj: 1.3076
03/19 23:27:14 - mmengine - INFO - Epoch(train)  [82][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:06:29  time: 0.2094  data_time: 0.0085  memory: 3937  loss: 4.2429  loss_cls: 0.6570  loss_bbox: 2.3004  loss_obj: 1.2855
03/19 23:27:25 - mmengine - INFO - Epoch(train)  [82][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:06:18  time: 0.2215  data_time: 0.0083  memory: 3937  loss: 4.2711  loss_cls: 0.6660  loss_bbox: 2.3013  loss_obj: 1.3038
03/19 23:27:35 - mmengine - INFO - Epoch(train)  [82][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:06:07  time: 0.1979  data_time: 0.0081  memory: 2587  loss: 4.2435  loss_cls: 0.6747  loss_bbox: 2.3077  loss_obj: 1.2611
03/19 23:27:46 - mmengine - INFO - Epoch(train)  [82][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:57  time: 0.2231  data_time: 0.0084  memory: 3639  loss: 4.3619  loss_cls: 0.6784  loss_bbox: 2.3396  loss_obj: 1.3439
03/19 23:27:58 - mmengine - INFO - Epoch(train)  [82][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:46  time: 0.2432  data_time: 0.0083  memory: 3937  loss: 4.2535  loss_cls: 0.6611  loss_bbox: 2.3020  loss_obj: 1.2904
03/19 23:28:09 - mmengine - INFO - Epoch(train)  [82][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:35  time: 0.2119  data_time: 0.0082  memory: 3937  loss: 4.3116  loss_cls: 0.6751  loss_bbox: 2.3263  loss_obj: 1.3102
03/19 23:28:20 - mmengine - INFO - Epoch(train)  [82][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:24  time: 0.2096  data_time: 0.0086  memory: 3639  loss: 4.3524  loss_cls: 0.6795  loss_bbox: 2.3563  loss_obj: 1.3167
03/19 23:28:30 - mmengine - INFO - Epoch(train)  [82][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:13  time: 0.2176  data_time: 0.0085  memory: 3639  loss: 4.3241  loss_cls: 0.6719  loss_bbox: 2.3483  loss_obj: 1.3039
03/19 23:28:42 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:28:42 - mmengine - INFO - Epoch(train)  [82][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:05:03  time: 0.2288  data_time: 0.0083  memory: 3937  loss: 4.2851  loss_cls: 0.6663  loss_bbox: 2.3207  loss_obj: 1.2981
03/19 23:28:42 - mmengine - INFO - Saving checkpoint at 82 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:28:47 - mmengine - INFO - Epoch(val)  [82][ 50/250]    eta: 0:00:11  time: 0.0587  data_time: 0.0071  memory: 527  
03/19 23:28:50 - mmengine - INFO - Epoch(val)  [82][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:28:53 - mmengine - INFO - Epoch(val)  [82][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0065  memory: 527  
03/19 23:28:56 - mmengine - INFO - Epoch(val)  [82][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 23:28:59 - mmengine - INFO - Epoch(val)  [82][250/250]    eta: 0:00:00  time: 0.0569  data_time: 0.0064  memory: 527  
03/19 23:29:00 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.24s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.26s).
Accumulating evaluation results...
DONE (t=1.88s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.240
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.581
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.152
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.164
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.299
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.487
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.348
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.289
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.533
03/19 23:29:10 - mmengine - INFO - bbox_mAP_copypaste: 0.240 0.581 0.152 0.164 0.299 0.487
03/19 23:29:10 - mmengine - INFO - Epoch(val) [82][250/250]    coco/bbox_mAP: 0.2400  coco/bbox_mAP_50: 0.5810  coco/bbox_mAP_75: 0.1520  coco/bbox_mAP_s: 0.1640  coco/bbox_mAP_m: 0.2990  coco/bbox_mAP_l: 0.4870  data_time: 0.0066  time: 0.0581
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:29:21 - mmengine - INFO - Epoch(train)  [83][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:04:52  time: 0.2291  data_time: 0.0189  memory: 3357  loss: 4.3156  loss_cls: 0.6736  loss_bbox: 2.3422  loss_obj: 1.2998
03/19 23:29:32 - mmengine - INFO - Epoch(train)  [83][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:04:41  time: 0.2201  data_time: 0.0082  memory: 3937  loss: 4.2625  loss_cls: 0.6593  loss_bbox: 2.3079  loss_obj: 1.2953
03/19 23:29:41 - mmengine - INFO - Epoch(train)  [83][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:04:30  time: 0.1819  data_time: 0.0087  memory: 2817  loss: 4.2880  loss_cls: 0.6709  loss_bbox: 2.3631  loss_obj: 1.2540
03/19 23:29:52 - mmengine - INFO - Epoch(train)  [83][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:04:19  time: 0.2170  data_time: 0.0082  memory: 3357  loss: 4.2807  loss_cls: 0.6692  loss_bbox: 2.3155  loss_obj: 1.2960
03/19 23:30:03 - mmengine - INFO - Epoch(train)  [83][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:04:08  time: 0.2174  data_time: 0.0083  memory: 3357  loss: 4.2208  loss_cls: 0.6550  loss_bbox: 2.3037  loss_obj: 1.2621
03/19 23:30:14 - mmengine - INFO - Epoch(train)  [83][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:58  time: 0.2245  data_time: 0.0082  memory: 3639  loss: 4.2761  loss_cls: 0.6583  loss_bbox: 2.3131  loss_obj: 1.3047
03/19 23:30:24 - mmengine - INFO - Epoch(train)  [83][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:46  time: 0.1874  data_time: 0.0083  memory: 2587  loss: 4.2931  loss_cls: 0.6685  loss_bbox: 2.3480  loss_obj: 1.2766
03/19 23:30:35 - mmengine - INFO - Epoch(train)  [83][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:36  time: 0.2242  data_time: 0.0083  memory: 3937  loss: 4.2835  loss_cls: 0.6648  loss_bbox: 2.3166  loss_obj: 1.3021
03/19 23:30:46 - mmengine - INFO - Epoch(train)  [83][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:25  time: 0.2194  data_time: 0.0084  memory: 3357  loss: 4.3160  loss_cls: 0.6723  loss_bbox: 2.3464  loss_obj: 1.2973
03/19 23:30:56 - mmengine - INFO - Epoch(train)  [83][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:14  time: 0.2038  data_time: 0.0083  memory: 2817  loss: 4.2712  loss_cls: 0.6751  loss_bbox: 2.3323  loss_obj: 1.2639
03/19 23:31:07 - mmengine - INFO - Epoch(train)  [83][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:03:03  time: 0.2174  data_time: 0.0082  memory: 3639  loss: 4.2566  loss_cls: 0.6634  loss_bbox: 2.3045  loss_obj: 1.2887
03/19 23:31:18 - mmengine - INFO - Epoch(train)  [83][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:02:52  time: 0.2234  data_time: 0.0084  memory: 3639  loss: 4.2747  loss_cls: 0.6647  loss_bbox: 2.3041  loss_obj: 1.3058
03/19 23:31:30 - mmengine - INFO - Epoch(train)  [83][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:02:41  time: 0.2270  data_time: 0.0083  memory: 3937  loss: 4.3159  loss_cls: 0.6733  loss_bbox: 2.3284  loss_obj: 1.3142
03/19 23:31:42 - mmengine - INFO - Epoch(train)  [83][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:02:31  time: 0.2418  data_time: 0.0082  memory: 3937  loss: 4.3235  loss_cls: 0.6664  loss_bbox: 2.3265  loss_obj: 1.3306
03/19 23:31:52 - mmengine - INFO - Epoch(train)  [83][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:02:20  time: 0.2100  data_time: 0.0085  memory: 3937  loss: 4.3668  loss_cls: 0.6707  loss_bbox: 2.3538  loss_obj: 1.3424
03/19 23:32:03 - mmengine - INFO - Epoch(train)  [83][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:02:09  time: 0.2083  data_time: 0.0083  memory: 2817  loss: 4.2810  loss_cls: 0.6684  loss_bbox: 2.3336  loss_obj: 1.2790
03/19 23:32:15 - mmengine - INFO - Epoch(train)  [83][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:59  time: 0.2492  data_time: 0.0083  memory: 3937  loss: 4.2872  loss_cls: 0.6666  loss_bbox: 2.3125  loss_obj: 1.3081
03/19 23:32:26 - mmengine - INFO - Epoch(train)  [83][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:48  time: 0.2124  data_time: 0.0083  memory: 3639  loss: 4.3000  loss_cls: 0.6701  loss_bbox: 2.3234  loss_obj: 1.3065
03/19 23:32:37 - mmengine - INFO - Epoch(train)  [83][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:37  time: 0.2216  data_time: 0.0082  memory: 3639  loss: 4.2184  loss_cls: 0.6580  loss_bbox: 2.2749  loss_obj: 1.2855
03/19 23:32:48 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:32:48 - mmengine - INFO - Epoch(train)  [83][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:26  time: 0.2135  data_time: 0.0082  memory: 2817  loss: 4.2659  loss_cls: 0.6645  loss_bbox: 2.3123  loss_obj: 1.2890
03/19 23:32:48 - mmengine - INFO - Saving checkpoint at 83 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:32:53 - mmengine - INFO - Epoch(val)  [83][ 50/250]    eta: 0:00:12  time: 0.0617  data_time: 0.0097  memory: 527  
03/19 23:32:56 - mmengine - INFO - Epoch(val)  [83][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0064  memory: 527  
03/19 23:32:59 - mmengine - INFO - Epoch(val)  [83][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/19 23:33:02 - mmengine - INFO - Epoch(val)  [83][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0065  memory: 527  
03/19 23:33:05 - mmengine - INFO - Epoch(val)  [83][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 23:33:06 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.09s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.24s).
Accumulating evaluation results...
DONE (t=1.86s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.151
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.163
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.298
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.485
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.347
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.347
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.347
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.288
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.394
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.533
03/19 23:33:16 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.577 0.151 0.163 0.298 0.485
03/19 23:33:16 - mmengine - INFO - Epoch(val) [83][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1510  coco/bbox_mAP_s: 0.1630  coco/bbox_mAP_m: 0.2980  coco/bbox_mAP_l: 0.4850  data_time: 0.0071  time: 0.0589
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:33:28 - mmengine - INFO - Epoch(train)  [84][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:15  time: 0.2394  data_time: 0.0194  memory: 3937  loss: 4.2989  loss_cls: 0.6747  loss_bbox: 2.3189  loss_obj: 1.3053
03/19 23:33:40 - mmengine - INFO - Epoch(train)  [84][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:01:05  time: 0.2493  data_time: 0.0083  memory: 3937  loss: 4.2558  loss_cls: 0.6522  loss_bbox: 2.3114  loss_obj: 1.2922
03/19 23:33:52 - mmengine - INFO - Epoch(train)  [84][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:54  time: 0.2295  data_time: 0.0085  memory: 3357  loss: 4.2930  loss_cls: 0.6652  loss_bbox: 2.3286  loss_obj: 1.2991
03/19 23:34:03 - mmengine - INFO - Epoch(train)  [84][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:43  time: 0.2323  data_time: 0.0083  memory: 3937  loss: 4.2544  loss_cls: 0.6624  loss_bbox: 2.3173  loss_obj: 1.2747
03/19 23:34:13 - mmengine - INFO - Epoch(train)  [84][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:32  time: 0.1877  data_time: 0.0085  memory: 2337  loss: 4.3294  loss_cls: 0.6708  loss_bbox: 2.3547  loss_obj: 1.3039
03/19 23:34:23 - mmengine - INFO - Epoch(train)  [84][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:21  time: 0.2108  data_time: 0.0084  memory: 3357  loss: 4.2623  loss_cls: 0.6699  loss_bbox: 2.3359  loss_obj: 1.2566
03/19 23:34:34 - mmengine - INFO - Epoch(train)  [84][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:11  time: 0.2221  data_time: 0.0082  memory: 3357  loss: 4.2782  loss_cls: 0.6637  loss_bbox: 2.3167  loss_obj: 1.2978
03/19 23:34:46 - mmengine - INFO - Epoch(train)  [84][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 1:00:00  time: 0.2244  data_time: 0.0083  memory: 3357  loss: 4.2747  loss_cls: 0.6745  loss_bbox: 2.3252  loss_obj: 1.2750
03/19 23:34:56 - mmengine - INFO - Epoch(train)  [84][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:59:49  time: 0.2150  data_time: 0.0084  memory: 3071  loss: 4.3135  loss_cls: 0.6750  loss_bbox: 2.3279  loss_obj: 1.3107
03/19 23:35:07 - mmengine - INFO - Epoch(train)  [84][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:59:38  time: 0.2098  data_time: 0.0084  memory: 3937  loss: 4.3527  loss_cls: 0.6782  loss_bbox: 2.3568  loss_obj: 1.3177
03/19 23:35:18 - mmengine - INFO - Epoch(train)  [84][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:59:27  time: 0.2144  data_time: 0.0088  memory: 3357  loss: 4.3017  loss_cls: 0.6711  loss_bbox: 2.3208  loss_obj: 1.3098
03/19 23:35:29 - mmengine - INFO - Epoch(train)  [84][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:59:16  time: 0.2302  data_time: 0.0084  memory: 3639  loss: 4.2017  loss_cls: 0.6615  loss_bbox: 2.2792  loss_obj: 1.2611
03/19 23:35:39 - mmengine - INFO - Epoch(train)  [84][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:59:05  time: 0.2024  data_time: 0.0093  memory: 3071  loss: 4.2527  loss_cls: 0.6721  loss_bbox: 2.3345  loss_obj: 1.2461
03/19 23:35:50 - mmengine - INFO - Epoch(train)  [84][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:55  time: 0.2108  data_time: 0.0082  memory: 3071  loss: 4.1936  loss_cls: 0.6673  loss_bbox: 2.2892  loss_obj: 1.2371
03/19 23:36:00 - mmengine - INFO - Epoch(train)  [84][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:44  time: 0.2124  data_time: 0.0083  memory: 3071  loss: 4.2624  loss_cls: 0.6684  loss_bbox: 2.3407  loss_obj: 1.2534
03/19 23:36:12 - mmengine - INFO - Epoch(train)  [84][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:33  time: 0.2226  data_time: 0.0084  memory: 3937  loss: 4.2716  loss_cls: 0.6615  loss_bbox: 2.3355  loss_obj: 1.2746
03/19 23:36:22 - mmengine - INFO - Epoch(train)  [84][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:22  time: 0.2175  data_time: 0.0083  memory: 3357  loss: 4.2920  loss_cls: 0.6682  loss_bbox: 2.3311  loss_obj: 1.2926
03/19 23:36:34 - mmengine - INFO - Epoch(train)  [84][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:11  time: 0.2283  data_time: 0.0084  memory: 3639  loss: 4.2876  loss_cls: 0.6740  loss_bbox: 2.3319  loss_obj: 1.2817
03/19 23:36:44 - mmengine - INFO - Epoch(train)  [84][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:58:00  time: 0.1965  data_time: 0.0083  memory: 2587  loss: 4.2756  loss_cls: 0.6761  loss_bbox: 2.3294  loss_obj: 1.2701
03/19 23:36:55 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:36:55 - mmengine - INFO - Epoch(train)  [84][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:57:50  time: 0.2346  data_time: 0.0083  memory: 3937  loss: 4.3198  loss_cls: 0.6723  loss_bbox: 2.3324  loss_obj: 1.3152
03/19 23:36:55 - mmengine - INFO - Saving checkpoint at 84 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:37:01 - mmengine - INFO - Epoch(val)  [84][ 50/250]    eta: 0:00:12  time: 0.0605  data_time: 0.0072  memory: 527  
03/19 23:37:04 - mmengine - INFO - Epoch(val)  [84][100/250]    eta: 0:00:09  time: 0.0602  data_time: 0.0064  memory: 527  
03/19 23:37:07 - mmengine - INFO - Epoch(val)  [84][150/250]    eta: 0:00:06  time: 0.0598  data_time: 0.0065  memory: 527  
03/19 23:37:10 - mmengine - INFO - Epoch(val)  [84][200/250]    eta: 0:00:03  time: 0.0597  data_time: 0.0064  memory: 527  
03/19 23:37:13 - mmengine - INFO - Epoch(val)  [84][250/250]    eta: 0:00:00  time: 0.0595  data_time: 0.0065  memory: 527  
03/19 23:37:14 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.25s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=7.42s).
Accumulating evaluation results...
DONE (t=1.92s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.239
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.577
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.162
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.298
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.483
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.345
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.288
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.532
03/19 23:37:24 - mmengine - INFO - bbox_mAP_copypaste: 0.239 0.577 0.150 0.162 0.298 0.483
03/19 23:37:24 - mmengine - INFO - Epoch(val) [84][250/250]    coco/bbox_mAP: 0.2390  coco/bbox_mAP_50: 0.5770  coco/bbox_mAP_75: 0.1500  coco/bbox_mAP_s: 0.1620  coco/bbox_mAP_m: 0.2980  coco/bbox_mAP_l: 0.4830  data_time: 0.0066  time: 0.0599
03/19 23:37:24 - mmengine - INFO - No mosaic and mixup aug now!
03/19 23:37:24 - mmengine - INFO - Add additional L1 loss now!
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:37:34 - mmengine - INFO - Epoch(train)  [85][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:57:39  time: 0.1959  data_time: 0.0117  memory: 3357  loss: 4.3035  loss_cls: 0.5882  loss_bbox: 2.1080  loss_obj: 0.9375  loss_l1: 0.6697
03/19 23:37:44 - mmengine - INFO - Epoch(train)  [85][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:57:28  time: 0.2030  data_time: 0.0078  memory: 3937  loss: 4.3249  loss_cls: 0.5981  loss_bbox: 2.1289  loss_obj: 0.9029  loss_l1: 0.6950
03/19 23:37:55 - mmengine - INFO - Epoch(train)  [85][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:57:17  time: 0.2135  data_time: 0.0077  memory: 3937  loss: 4.3101  loss_cls: 0.5850  loss_bbox: 2.1033  loss_obj: 0.9248  loss_l1: 0.6970
03/19 23:38:05 - mmengine - INFO - Epoch(train)  [85][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:57:06  time: 0.2100  data_time: 0.0077  memory: 3639  loss: 4.4497  loss_cls: 0.5961  loss_bbox: 2.1620  loss_obj: 0.9815  loss_l1: 0.7101
03/19 23:38:15 - mmengine - INFO - Epoch(train)  [85][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:55  time: 0.1879  data_time: 0.0076  memory: 3071  loss: 4.3672  loss_cls: 0.5931  loss_bbox: 2.1472  loss_obj: 0.9486  loss_l1: 0.6783
03/19 23:38:25 - mmengine - INFO - Epoch(train)  [85][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:44  time: 0.2032  data_time: 0.0076  memory: 3937  loss: 4.3561  loss_cls: 0.5934  loss_bbox: 2.1475  loss_obj: 0.9206  loss_l1: 0.6947
03/19 23:38:36 - mmengine - INFO - Epoch(train)  [85][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:33  time: 0.2271  data_time: 0.0076  memory: 3639  loss: 4.3181  loss_cls: 0.5796  loss_bbox: 2.0885  loss_obj: 0.9183  loss_l1: 0.7317
03/19 23:38:46 - mmengine - INFO - Epoch(train)  [85][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:22  time: 0.1849  data_time: 0.0076  memory: 2587  loss: 4.2939  loss_cls: 0.5934  loss_bbox: 2.1173  loss_obj: 0.9113  loss_l1: 0.6720
03/19 23:38:55 - mmengine - INFO - Epoch(train)  [85][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:11  time: 0.1787  data_time: 0.0076  memory: 3357  loss: 4.3688  loss_cls: 0.5969  loss_bbox: 2.1607  loss_obj: 0.9419  loss_l1: 0.6694
03/19 23:39:05 - mmengine - INFO - Epoch(train)  [85][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:56:00  time: 0.2078  data_time: 0.0076  memory: 3937  loss: 4.3679  loss_cls: 0.5930  loss_bbox: 2.1310  loss_obj: 0.9365  loss_l1: 0.7073
03/19 23:39:15 - mmengine - INFO - Epoch(train)  [85][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:55:49  time: 0.1954  data_time: 0.0076  memory: 3937  loss: 4.3956  loss_cls: 0.6002  loss_bbox: 2.1502  loss_obj: 0.9522  loss_l1: 0.6931
03/19 23:39:24 - mmengine - INFO - Epoch(train)  [85][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:55:38  time: 0.1914  data_time: 0.0076  memory: 3357  loss: 4.3716  loss_cls: 0.5989  loss_bbox: 2.1347  loss_obj: 0.9437  loss_l1: 0.6944
03/19 23:39:36 - mmengine - INFO - Epoch(train)  [85][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:55:27  time: 0.2271  data_time: 0.0076  memory: 3937  loss: 4.4676  loss_cls: 0.5916  loss_bbox: 2.1685  loss_obj: 0.9522  loss_l1: 0.7553
03/19 23:39:47 - mmengine - INFO - Epoch(train)  [85][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:55:16  time: 0.2180  data_time: 0.0076  memory: 3937  loss: 4.2860  loss_cls: 0.5806  loss_bbox: 2.1018  loss_obj: 0.8886  loss_l1: 0.7151
03/19 23:39:58 - mmengine - INFO - Epoch(train)  [85][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:55:05  time: 0.2248  data_time: 0.0076  memory: 3937  loss: 4.3588  loss_cls: 0.5848  loss_bbox: 2.1148  loss_obj: 0.9379  loss_l1: 0.7213
03/19 23:40:07 - mmengine - INFO - Epoch(train)  [85][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:54:54  time: 0.1799  data_time: 0.0076  memory: 3357  loss: 4.3896  loss_cls: 0.5976  loss_bbox: 2.1495  loss_obj: 0.9779  loss_l1: 0.6645
03/19 23:40:17 - mmengine - INFO - Epoch(train)  [85][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:54:43  time: 0.1971  data_time: 0.0077  memory: 3357  loss: 4.2714  loss_cls: 0.5843  loss_bbox: 2.0961  loss_obj: 0.9121  loss_l1: 0.6788
03/19 23:40:27 - mmengine - INFO - Epoch(train)  [85][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:54:32  time: 0.2103  data_time: 0.0076  memory: 3639  loss: 4.3854  loss_cls: 0.5996  loss_bbox: 2.1382  loss_obj: 0.9374  loss_l1: 0.7102
03/19 23:40:38 - mmengine - INFO - Epoch(train)  [85][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:54:21  time: 0.2065  data_time: 0.0075  memory: 3937  loss: 4.3560  loss_cls: 0.5870  loss_bbox: 2.1274  loss_obj: 0.9454  loss_l1: 0.6962
03/19 23:40:47 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:40:47 - mmengine - INFO - Epoch(train)  [85][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:54:10  time: 0.1970  data_time: 0.0076  memory: 3071  loss: 4.2077  loss_cls: 0.5831  loss_bbox: 2.0825  loss_obj: 0.8603  loss_l1: 0.6818
03/19 23:40:47 - mmengine - INFO - Saving checkpoint at 85 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:40:53 - mmengine - INFO - Epoch(val)  [85][ 50/250]    eta: 0:00:11  time: 0.0600  data_time: 0.0072  memory: 527  
03/19 23:40:56 - mmengine - INFO - Epoch(val)  [85][100/250]    eta: 0:00:08  time: 0.0579  data_time: 0.0063  memory: 527  
03/19 23:40:59 - mmengine - INFO - Epoch(val)  [85][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0063  memory: 527  
03/19 23:41:02 - mmengine - INFO - Epoch(val)  [85][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0065  memory: 527  
03/19 23:41:05 - mmengine - INFO - Epoch(val)  [85][250/250]    eta: 0:00:00  time: 0.0563  data_time: 0.0063  memory: 527  
03/19 23:41:05 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.22s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=6.09s).
Accumulating evaluation results...
DONE (t=1.49s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.253
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.589
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.168
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.176
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.314
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.495
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.355
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.355
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.355
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.296
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.408
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.540
03/19 23:41:13 - mmengine - INFO - bbox_mAP_copypaste: 0.253 0.589 0.168 0.176 0.314 0.495
03/19 23:41:14 - mmengine - INFO - Epoch(val) [85][250/250]    coco/bbox_mAP: 0.2530  coco/bbox_mAP_50: 0.5890  coco/bbox_mAP_75: 0.1680  coco/bbox_mAP_s: 0.1760  coco/bbox_mAP_m: 0.3140  coco/bbox_mAP_l: 0.4950  data_time: 0.0065  time: 0.0582
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:41:24 - mmengine - INFO - Epoch(train)  [86][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:59  time: 0.2009  data_time: 0.0094  memory: 3937  loss: 4.1793  loss_cls: 0.5736  loss_bbox: 2.0511  loss_obj: 0.8956  loss_l1: 0.6591
03/19 23:41:34 - mmengine - INFO - Epoch(train)  [86][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:48  time: 0.2153  data_time: 0.0075  memory: 3357  loss: 4.1879  loss_cls: 0.5726  loss_bbox: 2.0582  loss_obj: 0.8798  loss_l1: 0.6773
03/19 23:41:44 - mmengine - INFO - Epoch(train)  [86][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:37  time: 0.1904  data_time: 0.0075  memory: 3937  loss: 4.1715  loss_cls: 0.5804  loss_bbox: 2.1033  loss_obj: 0.8281  loss_l1: 0.6597
03/19 23:41:54 - mmengine - INFO - Epoch(train)  [86][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:26  time: 0.1935  data_time: 0.0075  memory: 3357  loss: 4.2850  loss_cls: 0.5889  loss_bbox: 2.0939  loss_obj: 0.9362  loss_l1: 0.6660
03/19 23:42:03 - mmengine - INFO - Epoch(train)  [86][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:15  time: 0.1968  data_time: 0.0075  memory: 3639  loss: 4.2698  loss_cls: 0.5836  loss_bbox: 2.1115  loss_obj: 0.8976  loss_l1: 0.6770
03/19 23:42:14 - mmengine - INFO - Epoch(train)  [86][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:53:04  time: 0.2053  data_time: 0.0075  memory: 3071  loss: 4.2647  loss_cls: 0.5853  loss_bbox: 2.0969  loss_obj: 0.8956  loss_l1: 0.6868
03/19 23:42:24 - mmengine - INFO - Epoch(train)  [86][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:52:53  time: 0.1987  data_time: 0.0076  memory: 3071  loss: 4.1462  loss_cls: 0.5756  loss_bbox: 2.0632  loss_obj: 0.8480  loss_l1: 0.6593
03/19 23:42:34 - mmengine - INFO - Epoch(train)  [86][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:52:42  time: 0.2024  data_time: 0.0076  memory: 3357  loss: 4.2507  loss_cls: 0.5880  loss_bbox: 2.0902  loss_obj: 0.8896  loss_l1: 0.6829
03/19 23:42:44 - mmengine - INFO - Epoch(train)  [86][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:52:32  time: 0.2124  data_time: 0.0076  memory: 3937  loss: 4.3356  loss_cls: 0.5861  loss_bbox: 2.1172  loss_obj: 0.9367  loss_l1: 0.6955
03/19 23:42:56 - mmengine - INFO - Epoch(train)  [86][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:52:21  time: 0.2283  data_time: 0.0075  memory: 3937  loss: 4.2047  loss_cls: 0.5780  loss_bbox: 2.0686  loss_obj: 0.8513  loss_l1: 0.7068
03/19 23:43:07 - mmengine - INFO - Epoch(train)  [86][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:52:10  time: 0.2131  data_time: 0.0076  memory: 3639  loss: 4.2979  loss_cls: 0.5838  loss_bbox: 2.1193  loss_obj: 0.8887  loss_l1: 0.7062
03/19 23:43:17 - mmengine - INFO - Epoch(train)  [86][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:59  time: 0.2003  data_time: 0.0076  memory: 3357  loss: 4.3671  loss_cls: 0.5968  loss_bbox: 2.1166  loss_obj: 0.9706  loss_l1: 0.6831
03/19 23:43:28 - mmengine - INFO - Epoch(train)  [86][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:48  time: 0.2216  data_time: 0.0075  memory: 3937  loss: 4.3143  loss_cls: 0.5903  loss_bbox: 2.1021  loss_obj: 0.9170  loss_l1: 0.7049
03/19 23:43:39 - mmengine - INFO - Epoch(train)  [86][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:37  time: 0.2204  data_time: 0.0076  memory: 3937  loss: 4.2627  loss_cls: 0.5831  loss_bbox: 2.0996  loss_obj: 0.8577  loss_l1: 0.7223
03/19 23:43:49 - mmengine - INFO - Epoch(train)  [86][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:26  time: 0.1991  data_time: 0.0076  memory: 3071  loss: 4.3378  loss_cls: 0.5834  loss_bbox: 2.1325  loss_obj: 0.9212  loss_l1: 0.7008
03/19 23:43:59 - mmengine - INFO - Epoch(train)  [86][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:16  time: 0.1989  data_time: 0.0076  memory: 2817  loss: 4.3211  loss_cls: 0.5878  loss_bbox: 2.1123  loss_obj: 0.9317  loss_l1: 0.6893
03/19 23:44:09 - mmengine - INFO - Epoch(train)  [86][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:51:05  time: 0.2058  data_time: 0.0077  memory: 3937  loss: 4.3397  loss_cls: 0.5911  loss_bbox: 2.1199  loss_obj: 0.9312  loss_l1: 0.6975
03/19 23:44:18 - mmengine - INFO - Epoch(train)  [86][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:50:53  time: 0.1738  data_time: 0.0075  memory: 2587  loss: 4.3043  loss_cls: 0.6056  loss_bbox: 2.1156  loss_obj: 0.9326  loss_l1: 0.6505
03/19 23:44:28 - mmengine - INFO - Epoch(train)  [86][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:50:42  time: 0.2034  data_time: 0.0075  memory: 3639  loss: 4.2681  loss_cls: 0.5825  loss_bbox: 2.1041  loss_obj: 0.8974  loss_l1: 0.6841
03/19 23:44:37 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:44:37 - mmengine - INFO - Epoch(train)  [86][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:50:31  time: 0.1918  data_time: 0.0076  memory: 3071  loss: 4.2458  loss_cls: 0.5899  loss_bbox: 2.0836  loss_obj: 0.9144  loss_l1: 0.6580
03/19 23:44:37 - mmengine - INFO - Saving checkpoint at 86 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:44:43 - mmengine - INFO - Epoch(val)  [86][ 50/250]    eta: 0:00:11  time: 0.0584  data_time: 0.0071  memory: 527  
03/19 23:44:46 - mmengine - INFO - Epoch(val)  [86][100/250]    eta: 0:00:08  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:44:49 - mmengine - INFO - Epoch(val)  [86][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/19 23:44:52 - mmengine - INFO - Epoch(val)  [86][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 23:44:54 - mmengine - INFO - Epoch(val)  [86][250/250]    eta: 0:00:00  time: 0.0575  data_time: 0.0064  memory: 527  
03/19 23:44:55 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.21s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=5.34s).
Accumulating evaluation results...
DONE (t=1.26s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.267
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.618
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.180
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.190
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.327
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.519
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.366
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.366
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.366
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.306
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.420
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.560
03/19 23:45:02 - mmengine - INFO - bbox_mAP_copypaste: 0.267 0.618 0.180 0.190 0.327 0.519
03/19 23:45:02 - mmengine - INFO - Epoch(val) [86][250/250]    coco/bbox_mAP: 0.2670  coco/bbox_mAP_50: 0.6180  coco/bbox_mAP_75: 0.1800  coco/bbox_mAP_s: 0.1900  coco/bbox_mAP_m: 0.3270  coco/bbox_mAP_l: 0.5190  data_time: 0.0066  time: 0.0582
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:45:12 - mmengine - INFO - Epoch(train)  [87][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:50:20  time: 0.1842  data_time: 0.0098  memory: 2587  loss: 4.1269  loss_cls: 0.5857  loss_bbox: 2.0570  loss_obj: 0.8494  loss_l1: 0.6348
03/19 23:45:23 - mmengine - INFO - Epoch(train)  [87][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:50:10  time: 0.2226  data_time: 0.0077  memory: 3937  loss: 4.2631  loss_cls: 0.5824  loss_bbox: 2.0832  loss_obj: 0.8983  loss_l1: 0.6992
03/19 23:45:33 - mmengine - INFO - Epoch(train)  [87][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:59  time: 0.1969  data_time: 0.0077  memory: 3639  loss: 4.2061  loss_cls: 0.5816  loss_bbox: 2.0870  loss_obj: 0.8815  loss_l1: 0.6561
03/19 23:45:43 - mmengine - INFO - Epoch(train)  [87][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:48  time: 0.2167  data_time: 0.0076  memory: 3639  loss: 4.1673  loss_cls: 0.5681  loss_bbox: 2.0578  loss_obj: 0.8695  loss_l1: 0.6719
03/19 23:45:52 - mmengine - INFO - Epoch(train)  [87][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:37  time: 0.1730  data_time: 0.0076  memory: 2337  loss: 4.2269  loss_cls: 0.5963  loss_bbox: 2.1085  loss_obj: 0.8775  loss_l1: 0.6446
03/19 23:46:03 - mmengine - INFO - Epoch(train)  [87][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:26  time: 0.2168  data_time: 0.0077  memory: 3937  loss: 4.1729  loss_cls: 0.5754  loss_bbox: 2.0582  loss_obj: 0.8601  loss_l1: 0.6793
03/19 23:46:14 - mmengine - INFO - Epoch(train)  [87][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:15  time: 0.2151  data_time: 0.0076  memory: 3937  loss: 4.2331  loss_cls: 0.5791  loss_bbox: 2.0966  loss_obj: 0.8671  loss_l1: 0.6903
03/19 23:46:24 - mmengine - INFO - Epoch(train)  [87][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:49:04  time: 0.2049  data_time: 0.0076  memory: 3639  loss: 4.2951  loss_cls: 0.5896  loss_bbox: 2.1135  loss_obj: 0.9082  loss_l1: 0.6838
03/19 23:46:34 - mmengine - INFO - Epoch(train)  [87][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:48:53  time: 0.1928  data_time: 0.0077  memory: 2817  loss: 4.1337  loss_cls: 0.5760  loss_bbox: 2.0354  loss_obj: 0.8773  loss_l1: 0.6450
03/19 23:46:44 - mmengine - INFO - Epoch(train)  [87][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:48:42  time: 0.2096  data_time: 0.0077  memory: 3937  loss: 4.2257  loss_cls: 0.5780  loss_bbox: 2.0847  loss_obj: 0.8627  loss_l1: 0.7003
03/19 23:46:53 - mmengine - INFO - Epoch(train)  [87][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:48:31  time: 0.1880  data_time: 0.0077  memory: 3639  loss: 4.2678  loss_cls: 0.5880  loss_bbox: 2.0913  loss_obj: 0.9339  loss_l1: 0.6547
03/19 23:47:03 - mmengine - INFO - Epoch(train)  [87][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:48:20  time: 0.1884  data_time: 0.0076  memory: 3639  loss: 4.2191  loss_cls: 0.5857  loss_bbox: 2.0595  loss_obj: 0.9297  loss_l1: 0.6443
03/19 23:47:13 - mmengine - INFO - Epoch(train)  [87][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:48:09  time: 0.1990  data_time: 0.0077  memory: 3937  loss: 4.2242  loss_cls: 0.5843  loss_bbox: 2.0833  loss_obj: 0.8892  loss_l1: 0.6674
03/19 23:47:24 - mmengine - INFO - Epoch(train)  [87][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:58  time: 0.2245  data_time: 0.0077  memory: 3937  loss: 4.1715  loss_cls: 0.5735  loss_bbox: 2.0486  loss_obj: 0.8604  loss_l1: 0.6890
03/19 23:47:34 - mmengine - INFO - Epoch(train)  [87][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:47  time: 0.2078  data_time: 0.0077  memory: 3357  loss: 4.2416  loss_cls: 0.5881  loss_bbox: 2.0615  loss_obj: 0.9200  loss_l1: 0.6721
03/19 23:47:44 - mmengine - INFO - Epoch(train)  [87][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:36  time: 0.1872  data_time: 0.0076  memory: 2817  loss: 4.2221  loss_cls: 0.5753  loss_bbox: 2.0969  loss_obj: 0.8892  loss_l1: 0.6607
03/19 23:47:56 - mmengine - INFO - Epoch(train)  [87][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:26  time: 0.2336  data_time: 0.0077  memory: 3937  loss: 4.3396  loss_cls: 0.5867  loss_bbox: 2.1127  loss_obj: 0.9117  loss_l1: 0.7285
03/19 23:48:04 - mmengine - INFO - Epoch(train)  [87][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:15  time: 0.1775  data_time: 0.0077  memory: 3357  loss: 4.1741  loss_cls: 0.5886  loss_bbox: 2.0746  loss_obj: 0.8907  loss_l1: 0.6201
03/19 23:48:15 - mmengine - INFO - Epoch(train)  [87][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:47:04  time: 0.2085  data_time: 0.0077  memory: 3937  loss: 4.2998  loss_cls: 0.5912  loss_bbox: 2.1124  loss_obj: 0.9041  loss_l1: 0.6921
03/19 23:48:25 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:48:25 - mmengine - INFO - Epoch(train)  [87][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:46:53  time: 0.2014  data_time: 0.0075  memory: 3357  loss: 4.2607  loss_cls: 0.5804  loss_bbox: 2.0886  loss_obj: 0.9188  loss_l1: 0.6729
03/19 23:48:25 - mmengine - INFO - Saving checkpoint at 87 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:48:30 - mmengine - INFO - Epoch(val)  [87][ 50/250]    eta: 0:00:11  time: 0.0583  data_time: 0.0071  memory: 527  
03/19 23:48:33 - mmengine - INFO - Epoch(val)  [87][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 23:48:36 - mmengine - INFO - Epoch(val)  [87][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 23:48:39 - mmengine - INFO - Epoch(val)  [87][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0064  memory: 527  
03/19 23:48:42 - mmengine - INFO - Epoch(val)  [87][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0065  memory: 527  
03/19 23:48:43 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.19s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.93s).
Accumulating evaluation results...
DONE (t=1.11s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.277
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.631
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.198
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.201
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.333
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.541
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.373
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.373
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.373
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.313
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.423
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.579
03/19 23:48:49 - mmengine - INFO - bbox_mAP_copypaste: 0.277 0.631 0.198 0.201 0.333 0.541
03/19 23:48:49 - mmengine - INFO - Epoch(val) [87][250/250]    coco/bbox_mAP: 0.2770  coco/bbox_mAP_50: 0.6310  coco/bbox_mAP_75: 0.1980  coco/bbox_mAP_s: 0.2010  coco/bbox_mAP_m: 0.3330  coco/bbox_mAP_l: 0.5410  data_time: 0.0066  time: 0.0581
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:48:58 - mmengine - INFO - Epoch(train)  [88][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:46:42  time: 0.1896  data_time: 0.0098  memory: 3639  loss: 4.0972  loss_cls: 0.5714  loss_bbox: 2.0574  loss_obj: 0.8336  loss_l1: 0.6347
03/19 23:49:07 - mmengine - INFO - Epoch(train)  [88][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:46:31  time: 0.1764  data_time: 0.0076  memory: 3639  loss: 4.2495  loss_cls: 0.5944  loss_bbox: 2.1099  loss_obj: 0.9057  loss_l1: 0.6395
03/19 23:49:16 - mmengine - INFO - Epoch(train)  [88][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:46:19  time: 0.1732  data_time: 0.0076  memory: 2337  loss: 4.1494  loss_cls: 0.5874  loss_bbox: 2.0752  loss_obj: 0.8663  loss_l1: 0.6205
03/19 23:49:27 - mmengine - INFO - Epoch(train)  [88][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:46:09  time: 0.2175  data_time: 0.0077  memory: 3639  loss: 4.2546  loss_cls: 0.5726  loss_bbox: 2.0838  loss_obj: 0.8975  loss_l1: 0.7006
03/19 23:49:36 - mmengine - INFO - Epoch(train)  [88][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:58  time: 0.1882  data_time: 0.0077  memory: 3937  loss: 4.1659  loss_cls: 0.5845  loss_bbox: 2.0727  loss_obj: 0.8693  loss_l1: 0.6394
03/19 23:49:45 - mmengine - INFO - Epoch(train)  [88][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:47  time: 0.1830  data_time: 0.0077  memory: 3071  loss: 4.1812  loss_cls: 0.5823  loss_bbox: 2.0925  loss_obj: 0.8699  loss_l1: 0.6365
03/19 23:49:55 - mmengine - INFO - Epoch(train)  [88][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:36  time: 0.1843  data_time: 0.0077  memory: 3071  loss: 4.1821  loss_cls: 0.5818  loss_bbox: 2.0731  loss_obj: 0.8992  loss_l1: 0.6281
03/19 23:50:07 - mmengine - INFO - Epoch(train)  [88][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:25  time: 0.2455  data_time: 0.0077  memory: 3937  loss: 4.1985  loss_cls: 0.5713  loss_bbox: 2.0502  loss_obj: 0.8656  loss_l1: 0.7114
03/19 23:50:19 - mmengine - INFO - Epoch(train)  [88][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:14  time: 0.2372  data_time: 0.0077  memory: 3937  loss: 4.2478  loss_cls: 0.5873  loss_bbox: 2.0783  loss_obj: 0.8790  loss_l1: 0.7032
03/19 23:50:28 - mmengine - INFO - Epoch(train)  [88][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:45:03  time: 0.1899  data_time: 0.0076  memory: 3639  loss: 4.1722  loss_cls: 0.5829  loss_bbox: 2.0587  loss_obj: 0.8883  loss_l1: 0.6424
03/19 23:50:38 - mmengine - INFO - Epoch(train)  [88][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:44:52  time: 0.1840  data_time: 0.0077  memory: 3357  loss: 4.1889  loss_cls: 0.5765  loss_bbox: 2.0621  loss_obj: 0.8960  loss_l1: 0.6541
03/19 23:50:47 - mmengine - INFO - Epoch(train)  [88][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:44:41  time: 0.1962  data_time: 0.0076  memory: 3639  loss: 4.1247  loss_cls: 0.5847  loss_bbox: 2.0435  loss_obj: 0.8594  loss_l1: 0.6371
03/19 23:50:57 - mmengine - INFO - Epoch(train)  [88][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:44:30  time: 0.1990  data_time: 0.0076  memory: 3357  loss: 4.2235  loss_cls: 0.5839  loss_bbox: 2.0720  loss_obj: 0.8990  loss_l1: 0.6686
03/19 23:51:06 - mmengine - INFO - Epoch(train)  [88][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:44:19  time: 0.1827  data_time: 0.0076  memory: 3071  loss: 4.0859  loss_cls: 0.5740  loss_bbox: 2.0424  loss_obj: 0.8469  loss_l1: 0.6227
03/19 23:51:17 - mmengine - INFO - Epoch(train)  [88][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:44:08  time: 0.2075  data_time: 0.0075  memory: 3639  loss: 4.2813  loss_cls: 0.5871  loss_bbox: 2.1032  loss_obj: 0.8988  loss_l1: 0.6921
03/19 23:51:27 - mmengine - INFO - Epoch(train)  [88][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:57  time: 0.1983  data_time: 0.0076  memory: 3937  loss: 4.2067  loss_cls: 0.5804  loss_bbox: 2.0751  loss_obj: 0.8984  loss_l1: 0.6528
03/19 23:51:36 - mmengine - INFO - Epoch(train)  [88][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:46  time: 0.1903  data_time: 0.0078  memory: 2817  loss: 4.1322  loss_cls: 0.5756  loss_bbox: 2.0341  loss_obj: 0.8902  loss_l1: 0.6322
03/19 23:51:45 - mmengine - INFO - Epoch(train)  [88][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:35  time: 0.1810  data_time: 0.0077  memory: 2587  loss: 4.0555  loss_cls: 0.5732  loss_bbox: 2.0272  loss_obj: 0.8574  loss_l1: 0.5977
03/19 23:51:55 - mmengine - INFO - Epoch(train)  [88][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:24  time: 0.1953  data_time: 0.0077  memory: 3357  loss: 4.1039  loss_cls: 0.5719  loss_bbox: 2.0513  loss_obj: 0.8376  loss_l1: 0.6430
03/19 23:52:05 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:52:05 - mmengine - INFO - Epoch(train)  [88][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:13  time: 0.1896  data_time: 0.0076  memory: 3639  loss: 4.1339  loss_cls: 0.5756  loss_bbox: 2.0705  loss_obj: 0.8425  loss_l1: 0.6454
03/19 23:52:05 - mmengine - INFO - Saving checkpoint at 88 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:52:10 - mmengine - INFO - Epoch(val)  [88][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0072  memory: 527  
03/19 23:52:13 - mmengine - INFO - Epoch(val)  [88][100/250]    eta: 0:00:08  time: 0.0581  data_time: 0.0064  memory: 527  
03/19 23:52:16 - mmengine - INFO - Epoch(val)  [88][150/250]    eta: 0:00:05  time: 0.0574  data_time: 0.0065  memory: 527  
03/19 23:52:19 - mmengine - INFO - Epoch(val)  [88][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 23:52:21 - mmengine - INFO - Epoch(val)  [88][250/250]    eta: 0:00:00  time: 0.0572  data_time: 0.0065  memory: 527  
03/19 23:52:22 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.19s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.48s).
Accumulating evaluation results...
DONE (t=1.22s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.282
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.631
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.202
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.207
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.335
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.535
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.376
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.376
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.376
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.316
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.421
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.578
03/19 23:52:28 - mmengine - INFO - bbox_mAP_copypaste: 0.282 0.631 0.202 0.207 0.335 0.535
03/19 23:52:28 - mmengine - INFO - Epoch(val) [88][250/250]    coco/bbox_mAP: 0.2820  coco/bbox_mAP_50: 0.6310  coco/bbox_mAP_75: 0.2020  coco/bbox_mAP_s: 0.2070  coco/bbox_mAP_m: 0.3350  coco/bbox_mAP_l: 0.5350  data_time: 0.0066  time: 0.0580
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:52:39 - mmengine - INFO - Epoch(train)  [89][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:43:03  time: 0.2206  data_time: 0.0096  memory: 3639  loss: 4.1610  loss_cls: 0.5694  loss_bbox: 2.0494  loss_obj: 0.8650  loss_l1: 0.6772
03/19 23:52:50 - mmengine - INFO - Epoch(train)  [89][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:42:52  time: 0.2072  data_time: 0.0077  memory: 3937  loss: 4.1445  loss_cls: 0.5709  loss_bbox: 2.0554  loss_obj: 0.8504  loss_l1: 0.6678
03/19 23:53:00 - mmengine - INFO - Epoch(train)  [89][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:42:41  time: 0.2057  data_time: 0.0077  memory: 3357  loss: 4.1715  loss_cls: 0.5696  loss_bbox: 2.0567  loss_obj: 0.8819  loss_l1: 0.6633
03/19 23:53:09 - mmengine - INFO - Epoch(train)  [89][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:42:30  time: 0.1860  data_time: 0.0078  memory: 3639  loss: 4.0854  loss_cls: 0.5779  loss_bbox: 2.0292  loss_obj: 0.8556  loss_l1: 0.6226
03/19 23:53:19 - mmengine - INFO - Epoch(train)  [89][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:42:19  time: 0.2007  data_time: 0.0076  memory: 3639  loss: 4.0053  loss_cls: 0.5638  loss_bbox: 2.0227  loss_obj: 0.7785  loss_l1: 0.6403
03/19 23:53:30 - mmengine - INFO - Epoch(train)  [89][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:42:08  time: 0.2032  data_time: 0.0077  memory: 3639  loss: 4.1189  loss_cls: 0.5749  loss_bbox: 2.0461  loss_obj: 0.8372  loss_l1: 0.6608
03/19 23:53:41 - mmengine - INFO - Epoch(train)  [89][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:57  time: 0.2268  data_time: 0.0077  memory: 3937  loss: 4.1577  loss_cls: 0.5702  loss_bbox: 2.0408  loss_obj: 0.8534  loss_l1: 0.6933
03/19 23:53:51 - mmengine - INFO - Epoch(train)  [89][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:47  time: 0.2063  data_time: 0.0077  memory: 3357  loss: 4.1539  loss_cls: 0.5795  loss_bbox: 2.0516  loss_obj: 0.8549  loss_l1: 0.6678
03/19 23:54:01 - mmengine - INFO - Epoch(train)  [89][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:36  time: 0.1879  data_time: 0.0077  memory: 2817  loss: 4.1451  loss_cls: 0.5820  loss_bbox: 2.0655  loss_obj: 0.8478  loss_l1: 0.6498
03/19 23:54:11 - mmengine - INFO - Epoch(train)  [89][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:25  time: 0.1982  data_time: 0.0077  memory: 3937  loss: 4.0944  loss_cls: 0.5720  loss_bbox: 2.0215  loss_obj: 0.8679  loss_l1: 0.6330
03/19 23:54:21 - mmengine - INFO - Epoch(train)  [89][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:14  time: 0.2019  data_time: 0.0076  memory: 3937  loss: 4.1316  loss_cls: 0.5685  loss_bbox: 2.0364  loss_obj: 0.8839  loss_l1: 0.6428
03/19 23:54:32 - mmengine - INFO - Epoch(train)  [89][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:41:03  time: 0.2200  data_time: 0.0078  memory: 3937  loss: 4.1397  loss_cls: 0.5723  loss_bbox: 2.0453  loss_obj: 0.8382  loss_l1: 0.6839
03/19 23:54:42 - mmengine - INFO - Epoch(train)  [89][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:40:52  time: 0.2061  data_time: 0.0077  memory: 3357  loss: 4.0884  loss_cls: 0.5628  loss_bbox: 2.0116  loss_obj: 0.8633  loss_l1: 0.6507
03/19 23:54:52 - mmengine - INFO - Epoch(train)  [89][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:40:41  time: 0.1910  data_time: 0.0077  memory: 3639  loss: 4.1772  loss_cls: 0.5825  loss_bbox: 2.0599  loss_obj: 0.8904  loss_l1: 0.6445
03/19 23:55:03 - mmengine - INFO - Epoch(train)  [89][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:40:30  time: 0.2230  data_time: 0.0077  memory: 3639  loss: 4.2284  loss_cls: 0.5782  loss_bbox: 2.0802  loss_obj: 0.8806  loss_l1: 0.6894
03/19 23:55:14 - mmengine - INFO - Epoch(train)  [89][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:40:20  time: 0.2166  data_time: 0.0077  memory: 3937  loss: 4.2753  loss_cls: 0.5748  loss_bbox: 2.0836  loss_obj: 0.9220  loss_l1: 0.6948
03/19 23:55:23 - mmengine - INFO - Epoch(train)  [89][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:40:09  time: 0.1800  data_time: 0.0077  memory: 2337  loss: 4.0612  loss_cls: 0.5786  loss_bbox: 2.0228  loss_obj: 0.8612  loss_l1: 0.5986
03/19 23:55:32 - mmengine - INFO - Epoch(train)  [89][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:58  time: 0.1802  data_time: 0.0077  memory: 2817  loss: 4.1771  loss_cls: 0.5803  loss_bbox: 2.0705  loss_obj: 0.8888  loss_l1: 0.6374
03/19 23:55:41 - mmengine - INFO - Epoch(train)  [89][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:47  time: 0.1985  data_time: 0.0077  memory: 3357  loss: 4.0371  loss_cls: 0.5674  loss_bbox: 2.0130  loss_obj: 0.8198  loss_l1: 0.6368
03/19 23:55:52 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:55:52 - mmengine - INFO - Epoch(train)  [89][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:36  time: 0.2098  data_time: 0.0076  memory: 3639  loss: 4.1765  loss_cls: 0.5771  loss_bbox: 2.0463  loss_obj: 0.8802  loss_l1: 0.6728
03/19 23:55:52 - mmengine - INFO - Saving checkpoint at 89 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:55:57 - mmengine - INFO - Epoch(val)  [89][ 50/250]    eta: 0:00:11  time: 0.0599  data_time: 0.0072  memory: 527  
03/19 23:56:00 - mmengine - INFO - Epoch(val)  [89][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/19 23:56:03 - mmengine - INFO - Epoch(val)  [89][150/250]    eta: 0:00:05  time: 0.0581  data_time: 0.0065  memory: 527  
03/19 23:56:06 - mmengine - INFO - Epoch(val)  [89][200/250]    eta: 0:00:02  time: 0.0582  data_time: 0.0064  memory: 527  
03/19 23:56:09 - mmengine - INFO - Epoch(val)  [89][250/250]    eta: 0:00:00  time: 0.0573  data_time: 0.0065  memory: 527  
03/19 23:56:10 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.19s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.27s).
Accumulating evaluation results...
DONE (t=0.98s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.289
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.640
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.211
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.213
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.344
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.548
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.380
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.380
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.380
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.322
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.427
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.589
03/19 23:56:15 - mmengine - INFO - bbox_mAP_copypaste: 0.289 0.640 0.211 0.213 0.344 0.548
03/19 23:56:15 - mmengine - INFO - Epoch(val) [89][250/250]    coco/bbox_mAP: 0.2890  coco/bbox_mAP_50: 0.6400  coco/bbox_mAP_75: 0.2110  coco/bbox_mAP_s: 0.2130  coco/bbox_mAP_m: 0.3440  coco/bbox_mAP_l: 0.5480  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:56:25 - mmengine - INFO - Epoch(train)  [90][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:25  time: 0.2041  data_time: 0.0096  memory: 3357  loss: 4.0086  loss_cls: 0.5743  loss_bbox: 1.9990  loss_obj: 0.8106  loss_l1: 0.6247
03/19 23:56:35 - mmengine - INFO - Epoch(train)  [90][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:14  time: 0.1910  data_time: 0.0077  memory: 3071  loss: 4.0500  loss_cls: 0.5805  loss_bbox: 2.0191  loss_obj: 0.8324  loss_l1: 0.6180
03/19 23:56:45 - mmengine - INFO - Epoch(train)  [90][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:39:03  time: 0.1933  data_time: 0.0076  memory: 3357  loss: 4.1106  loss_cls: 0.5760  loss_bbox: 2.0508  loss_obj: 0.8427  loss_l1: 0.6410
03/19 23:56:55 - mmengine - INFO - Epoch(train)  [90][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:38:52  time: 0.1992  data_time: 0.0077  memory: 3937  loss: 4.0947  loss_cls: 0.5722  loss_bbox: 2.0366  loss_obj: 0.8485  loss_l1: 0.6373
03/19 23:57:04 - mmengine - INFO - Epoch(train)  [90][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:38:41  time: 0.1878  data_time: 0.0077  memory: 3937  loss: 4.1068  loss_cls: 0.5719  loss_bbox: 2.0268  loss_obj: 0.8887  loss_l1: 0.6193
03/19 23:57:14 - mmengine - INFO - Epoch(train)  [90][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:38:30  time: 0.1910  data_time: 0.0076  memory: 3639  loss: 4.1278  loss_cls: 0.5762  loss_bbox: 2.0540  loss_obj: 0.8553  loss_l1: 0.6423
03/19 23:57:23 - mmengine - INFO - Epoch(train)  [90][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:38:19  time: 0.1883  data_time: 0.0076  memory: 3357  loss: 4.0592  loss_cls: 0.5736  loss_bbox: 2.0291  loss_obj: 0.8295  loss_l1: 0.6270
03/19 23:57:34 - mmengine - INFO - Epoch(train)  [90][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:38:08  time: 0.2214  data_time: 0.0077  memory: 3639  loss: 4.2205  loss_cls: 0.5752  loss_bbox: 2.0756  loss_obj: 0.8776  loss_l1: 0.6921
03/19 23:57:43 - mmengine - INFO - Epoch(train)  [90][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:57  time: 0.1778  data_time: 0.0076  memory: 3357  loss: 4.0289  loss_cls: 0.5761  loss_bbox: 1.9902  loss_obj: 0.8599  loss_l1: 0.6027
03/19 23:57:54 - mmengine - INFO - Epoch(train)  [90][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:47  time: 0.2182  data_time: 0.0077  memory: 3937  loss: 4.2014  loss_cls: 0.5745  loss_bbox: 2.0551  loss_obj: 0.8973  loss_l1: 0.6745
03/19 23:58:04 - mmengine - INFO - Epoch(train)  [90][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:36  time: 0.1954  data_time: 0.0077  memory: 3071  loss: 4.0667  loss_cls: 0.5671  loss_bbox: 2.0159  loss_obj: 0.8553  loss_l1: 0.6285
03/19 23:58:13 - mmengine - INFO - Epoch(train)  [90][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:25  time: 0.1944  data_time: 0.0076  memory: 3357  loss: 4.0673  loss_cls: 0.5739  loss_bbox: 2.0297  loss_obj: 0.8355  loss_l1: 0.6282
03/19 23:58:23 - mmengine - INFO - Epoch(train)  [90][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:14  time: 0.1952  data_time: 0.0076  memory: 3639  loss: 4.0790  loss_cls: 0.5636  loss_bbox: 2.0369  loss_obj: 0.8438  loss_l1: 0.6347
03/19 23:58:34 - mmengine - INFO - Epoch(train)  [90][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:37:03  time: 0.2066  data_time: 0.0076  memory: 3937  loss: 4.1783  loss_cls: 0.5857  loss_bbox: 2.0577  loss_obj: 0.8757  loss_l1: 0.6592
03/19 23:58:43 - mmengine - INFO - Epoch(train)  [90][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:36:52  time: 0.1933  data_time: 0.0076  memory: 3357  loss: 4.0759  loss_cls: 0.5654  loss_bbox: 2.0418  loss_obj: 0.8338  loss_l1: 0.6350
03/19 23:58:53 - mmengine - INFO - Epoch(train)  [90][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:36:41  time: 0.1954  data_time: 0.0078  memory: 3357  loss: 4.0509  loss_cls: 0.5628  loss_bbox: 2.0283  loss_obj: 0.8325  loss_l1: 0.6274
03/19 23:59:02 - mmengine - INFO - Epoch(train)  [90][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:36:30  time: 0.1761  data_time: 0.0077  memory: 2587  loss: 4.0170  loss_cls: 0.5624  loss_bbox: 2.0224  loss_obj: 0.8371  loss_l1: 0.5951
03/19 23:59:11 - mmengine - INFO - Epoch(train)  [90][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:36:19  time: 0.1784  data_time: 0.0078  memory: 2587  loss: 4.0111  loss_cls: 0.5690  loss_bbox: 1.9996  loss_obj: 0.8416  loss_l1: 0.6010
03/19 23:59:22 - mmengine - INFO - Epoch(train)  [90][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:36:08  time: 0.2190  data_time: 0.0077  memory: 3937  loss: 4.0865  loss_cls: 0.5793  loss_bbox: 2.0123  loss_obj: 0.8318  loss_l1: 0.6631
03/19 23:59:31 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/19 23:59:31 - mmengine - INFO - Epoch(train)  [90][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:58  time: 0.1953  data_time: 0.0077  memory: 3639  loss: 4.0245  loss_cls: 0.5684  loss_bbox: 1.9919  loss_obj: 0.8419  loss_l1: 0.6223
03/19 23:59:31 - mmengine - INFO - Saving checkpoint at 90 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/19 23:59:37 - mmengine - INFO - Epoch(val)  [90][ 50/250]    eta: 0:00:11  time: 0.0589  data_time: 0.0070  memory: 527  
03/19 23:59:40 - mmengine - INFO - Epoch(val)  [90][100/250]    eta: 0:00:08  time: 0.0584  data_time: 0.0064  memory: 527  
03/19 23:59:43 - mmengine - INFO - Epoch(val)  [90][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0064  memory: 527  
03/19 23:59:46 - mmengine - INFO - Epoch(val)  [90][200/250]    eta: 0:00:02  time: 0.0583  data_time: 0.0064  memory: 527  
03/19 23:59:48 - mmengine - INFO - Epoch(val)  [90][250/250]    eta: 0:00:00  time: 0.0561  data_time: 0.0064  memory: 527  
03/19 23:59:49 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.18s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.36s).
Accumulating evaluation results...
DONE (t=0.95s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.296
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.644
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.220
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.221
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.349
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.529
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.388
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.388
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.388
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.329
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.433
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.593
03/19 23:59:55 - mmengine - INFO - bbox_mAP_copypaste: 0.296 0.644 0.220 0.221 0.349 0.529
03/19 23:59:55 - mmengine - INFO - Epoch(val) [90][250/250]    coco/bbox_mAP: 0.2960  coco/bbox_mAP_50: 0.6440  coco/bbox_mAP_75: 0.2200  coco/bbox_mAP_s: 0.2210  coco/bbox_mAP_m: 0.3490  coco/bbox_mAP_l: 0.5290  data_time: 0.0065  time: 0.0579
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:00:05 - mmengine - INFO - Epoch(train)  [91][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:47  time: 0.1988  data_time: 0.0098  memory: 3357  loss: 3.9019  loss_cls: 0.5510  loss_bbox: 1.9425  loss_obj: 0.8155  loss_l1: 0.5928
03/20 00:00:15 - mmengine - INFO - Epoch(train)  [91][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:36  time: 0.2175  data_time: 0.0077  memory: 3937  loss: 4.1307  loss_cls: 0.5730  loss_bbox: 2.0203  loss_obj: 0.8745  loss_l1: 0.6629
03/20 00:00:26 - mmengine - INFO - Epoch(train)  [91][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:25  time: 0.2157  data_time: 0.0077  memory: 3937  loss: 3.9525  loss_cls: 0.5509  loss_bbox: 1.9548  loss_obj: 0.8044  loss_l1: 0.6424
03/20 00:00:36 - mmengine - INFO - Epoch(train)  [91][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:14  time: 0.2025  data_time: 0.0076  memory: 3937  loss: 4.1087  loss_cls: 0.5724  loss_bbox: 2.0467  loss_obj: 0.8422  loss_l1: 0.6474
03/20 00:00:47 - mmengine - INFO - Epoch(train)  [91][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:35:03  time: 0.2064  data_time: 0.0077  memory: 3937  loss: 4.0604  loss_cls: 0.5713  loss_bbox: 2.0300  loss_obj: 0.8133  loss_l1: 0.6459
03/20 00:00:59 - mmengine - INFO - Epoch(train)  [91][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:34:53  time: 0.2403  data_time: 0.0077  memory: 3937  loss: 4.1240  loss_cls: 0.5605  loss_bbox: 2.0284  loss_obj: 0.8353  loss_l1: 0.6997
03/20 00:01:08 - mmengine - INFO - Epoch(train)  [91][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:34:42  time: 0.1881  data_time: 0.0078  memory: 3357  loss: 4.0469  loss_cls: 0.5814  loss_bbox: 2.0273  loss_obj: 0.8249  loss_l1: 0.6132
03/20 00:01:17 - mmengine - INFO - Epoch(train)  [91][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:34:31  time: 0.1839  data_time: 0.0077  memory: 3071  loss: 4.0910  loss_cls: 0.5750  loss_bbox: 2.0319  loss_obj: 0.8653  loss_l1: 0.6187
03/20 00:01:26 - mmengine - INFO - Epoch(train)  [91][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:34:20  time: 0.1812  data_time: 0.0076  memory: 2817  loss: 3.9750  loss_cls: 0.5701  loss_bbox: 1.9967  loss_obj: 0.8083  loss_l1: 0.5999
03/20 00:01:37 - mmengine - INFO - Epoch(train)  [91][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:34:09  time: 0.2109  data_time: 0.0076  memory: 3937  loss: 4.0994  loss_cls: 0.5691  loss_bbox: 2.0511  loss_obj: 0.8135  loss_l1: 0.6657
03/20 00:01:49 - mmengine - INFO - Epoch(train)  [91][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:58  time: 0.2302  data_time: 0.0077  memory: 3937  loss: 4.0466  loss_cls: 0.5641  loss_bbox: 2.0116  loss_obj: 0.7953  loss_l1: 0.6757
03/20 00:01:59 - mmengine - INFO - Epoch(train)  [91][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:47  time: 0.2087  data_time: 0.0077  memory: 3937  loss: 4.0643  loss_cls: 0.5736  loss_bbox: 2.0149  loss_obj: 0.8325  loss_l1: 0.6434
03/20 00:02:08 - mmengine - INFO - Epoch(train)  [91][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:36  time: 0.1737  data_time: 0.0076  memory: 3071  loss: 4.0308  loss_cls: 0.5811  loss_bbox: 2.0325  loss_obj: 0.8096  loss_l1: 0.6075
03/20 00:02:18 - mmengine - INFO - Epoch(train)  [91][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:26  time: 0.1963  data_time: 0.0076  memory: 3639  loss: 4.0097  loss_cls: 0.5656  loss_bbox: 1.9896  loss_obj: 0.8281  loss_l1: 0.6264
03/20 00:02:27 - mmengine - INFO - Epoch(train)  [91][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:15  time: 0.1963  data_time: 0.0077  memory: 3639  loss: 4.0987  loss_cls: 0.5763  loss_bbox: 2.0408  loss_obj: 0.8336  loss_l1: 0.6480
03/20 00:02:37 - mmengine - INFO - Epoch(train)  [91][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:33:04  time: 0.1938  data_time: 0.0081  memory: 3071  loss: 4.0730  loss_cls: 0.5687  loss_bbox: 2.0304  loss_obj: 0.8548  loss_l1: 0.6193
03/20 00:02:47 - mmengine - INFO - Epoch(train)  [91][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:32:53  time: 0.1972  data_time: 0.0078  memory: 3639  loss: 3.9803  loss_cls: 0.5621  loss_bbox: 1.9852  loss_obj: 0.8230  loss_l1: 0.6101
03/20 00:02:57 - mmengine - INFO - Epoch(train)  [91][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:32:42  time: 0.2098  data_time: 0.0077  memory: 3639  loss: 4.1340  loss_cls: 0.5694  loss_bbox: 2.0514  loss_obj: 0.8450  loss_l1: 0.6681
03/20 00:03:09 - mmengine - INFO - Epoch(train)  [91][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:32:31  time: 0.2231  data_time: 0.0076  memory: 3937  loss: 4.1897  loss_cls: 0.5777  loss_bbox: 2.0581  loss_obj: 0.8683  loss_l1: 0.6856
03/20 00:03:18 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:03:18 - mmengine - INFO - Epoch(train)  [91][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:32:20  time: 0.1899  data_time: 0.0077  memory: 3357  loss: 4.1168  loss_cls: 0.5771  loss_bbox: 2.0516  loss_obj: 0.8556  loss_l1: 0.6325
03/20 00:03:18 - mmengine - INFO - Saving checkpoint at 91 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:03:23 - mmengine - INFO - Epoch(val)  [91][ 50/250]    eta: 0:00:11  time: 0.0598  data_time: 0.0074  memory: 527  
03/20 00:03:26 - mmengine - INFO - Epoch(val)  [91][100/250]    eta: 0:00:08  time: 0.0580  data_time: 0.0066  memory: 527  
03/20 00:03:29 - mmengine - INFO - Epoch(val)  [91][150/250]    eta: 0:00:05  time: 0.0589  data_time: 0.0066  memory: 527  
03/20 00:03:32 - mmengine - INFO - Epoch(val)  [91][200/250]    eta: 0:00:02  time: 0.0586  data_time: 0.0064  memory: 527  
03/20 00:03:35 - mmengine - INFO - Epoch(val)  [91][250/250]    eta: 0:00:00  time: 0.0568  data_time: 0.0065  memory: 527  
03/20 00:03:36 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.04s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.22s).
Accumulating evaluation results...
DONE (t=0.91s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.299
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.660
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.218
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.226
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.353
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.534
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.391
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.438
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.595
03/20 00:03:41 - mmengine - INFO - bbox_mAP_copypaste: 0.299 0.660 0.218 0.226 0.353 0.534
03/20 00:03:41 - mmengine - INFO - Epoch(val) [91][250/250]    coco/bbox_mAP: 0.2990  coco/bbox_mAP_50: 0.6600  coco/bbox_mAP_75: 0.2180  coco/bbox_mAP_s: 0.2260  coco/bbox_mAP_m: 0.3530  coco/bbox_mAP_l: 0.5340  data_time: 0.0067  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:03:52 - mmengine - INFO - Epoch(train)  [92][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:32:10  time: 0.2119  data_time: 0.0097  memory: 3639  loss: 4.0377  loss_cls: 0.5602  loss_bbox: 2.0218  loss_obj: 0.7989  loss_l1: 0.6568
03/20 00:04:02 - mmengine - INFO - Epoch(train)  [92][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:59  time: 0.2037  data_time: 0.0076  memory: 3639  loss: 4.0424  loss_cls: 0.5714  loss_bbox: 2.0119  loss_obj: 0.8212  loss_l1: 0.6379
03/20 00:04:12 - mmengine - INFO - Epoch(train)  [92][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:48  time: 0.2103  data_time: 0.0076  memory: 3937  loss: 3.9377  loss_cls: 0.5546  loss_bbox: 1.9665  loss_obj: 0.7899  loss_l1: 0.6268
03/20 00:04:22 - mmengine - INFO - Epoch(train)  [92][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:37  time: 0.2007  data_time: 0.0077  memory: 3071  loss: 4.0068  loss_cls: 0.5649  loss_bbox: 2.0096  loss_obj: 0.8154  loss_l1: 0.6170
03/20 00:04:33 - mmengine - INFO - Epoch(train)  [92][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:26  time: 0.2073  data_time: 0.0077  memory: 3639  loss: 4.0387  loss_cls: 0.5594  loss_bbox: 2.0020  loss_obj: 0.8442  loss_l1: 0.6332
03/20 00:04:43 - mmengine - INFO - Epoch(train)  [92][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:15  time: 0.2017  data_time: 0.0077  memory: 3937  loss: 3.9842  loss_cls: 0.5606  loss_bbox: 2.0016  loss_obj: 0.8017  loss_l1: 0.6202
03/20 00:04:54 - mmengine - INFO - Epoch(train)  [92][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:31:05  time: 0.2181  data_time: 0.0077  memory: 3937  loss: 4.0340  loss_cls: 0.5631  loss_bbox: 1.9940  loss_obj: 0.8336  loss_l1: 0.6433
03/20 00:05:04 - mmengine - INFO - Epoch(train)  [92][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:54  time: 0.2032  data_time: 0.0077  memory: 3071  loss: 4.0298  loss_cls: 0.5671  loss_bbox: 1.9919  loss_obj: 0.8366  loss_l1: 0.6342
03/20 00:05:14 - mmengine - INFO - Epoch(train)  [92][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:43  time: 0.1984  data_time: 0.0077  memory: 3937  loss: 3.9167  loss_cls: 0.5524  loss_bbox: 1.9648  loss_obj: 0.7911  loss_l1: 0.6084
03/20 00:05:24 - mmengine - INFO - Epoch(train)  [92][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:32  time: 0.2019  data_time: 0.0076  memory: 3937  loss: 4.0576  loss_cls: 0.5749  loss_bbox: 2.0350  loss_obj: 0.8123  loss_l1: 0.6354
03/20 00:05:34 - mmengine - INFO - Epoch(train)  [92][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:21  time: 0.1978  data_time: 0.0078  memory: 3639  loss: 3.9926  loss_cls: 0.5674  loss_bbox: 1.9920  loss_obj: 0.8154  loss_l1: 0.6178
03/20 00:05:45 - mmengine - INFO - Epoch(train)  [92][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:11  time: 0.2222  data_time: 0.0077  memory: 3639  loss: 4.0152  loss_cls: 0.5636  loss_bbox: 1.9868  loss_obj: 0.7913  loss_l1: 0.6736
03/20 00:05:55 - mmengine - INFO - Epoch(train)  [92][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:30:00  time: 0.2074  data_time: 0.0078  memory: 3639  loss: 4.0141  loss_cls: 0.5599  loss_bbox: 1.9944  loss_obj: 0.8215  loss_l1: 0.6383
03/20 00:06:05 - mmengine - INFO - Epoch(train)  [92][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:29:49  time: 0.1983  data_time: 0.0077  memory: 3071  loss: 3.9592  loss_cls: 0.5593  loss_bbox: 1.9792  loss_obj: 0.8093  loss_l1: 0.6114
03/20 00:06:16 - mmengine - INFO - Epoch(train)  [92][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:29:38  time: 0.2088  data_time: 0.0077  memory: 3639  loss: 4.0609  loss_cls: 0.5673  loss_bbox: 2.0301  loss_obj: 0.8118  loss_l1: 0.6517
03/20 00:06:26 - mmengine - INFO - Epoch(train)  [92][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:29:27  time: 0.1946  data_time: 0.0077  memory: 3937  loss: 4.0260  loss_cls: 0.5720  loss_bbox: 2.0086  loss_obj: 0.8234  loss_l1: 0.6221
03/20 00:06:35 - mmengine - INFO - Epoch(train)  [92][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:29:16  time: 0.1985  data_time: 0.0077  memory: 3357  loss: 3.9984  loss_cls: 0.5581  loss_bbox: 2.0109  loss_obj: 0.7954  loss_l1: 0.6340
03/20 00:06:44 - mmengine - INFO - Epoch(train)  [92][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:29:05  time: 0.1766  data_time: 0.0077  memory: 2817  loss: 3.9995  loss_cls: 0.5684  loss_bbox: 2.0091  loss_obj: 0.8319  loss_l1: 0.5901
03/20 00:06:54 - mmengine - INFO - Epoch(train)  [92][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:55  time: 0.2028  data_time: 0.0077  memory: 3639  loss: 4.1511  loss_cls: 0.5823  loss_bbox: 2.0469  loss_obj: 0.8647  loss_l1: 0.6571
03/20 00:07:04 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:07:04 - mmengine - INFO - Epoch(train)  [92][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:44  time: 0.1907  data_time: 0.0076  memory: 3937  loss: 4.0000  loss_cls: 0.5743  loss_bbox: 1.9955  loss_obj: 0.8328  loss_l1: 0.5974
03/20 00:07:04 - mmengine - INFO - Saving checkpoint at 92 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:07:09 - mmengine - INFO - Epoch(val)  [92][ 50/250]    eta: 0:00:11  time: 0.0594  data_time: 0.0072  memory: 527  
03/20 00:07:12 - mmengine - INFO - Epoch(val)  [92][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0065  memory: 527  
03/20 00:07:15 - mmengine - INFO - Epoch(val)  [92][150/250]    eta: 0:00:05  time: 0.0586  data_time: 0.0065  memory: 527  
03/20 00:07:18 - mmengine - INFO - Epoch(val)  [92][200/250]    eta: 0:00:02  time: 0.0578  data_time: 0.0064  memory: 527  
03/20 00:07:21 - mmengine - INFO - Epoch(val)  [92][250/250]    eta: 0:00:00  time: 0.0567  data_time: 0.0065  memory: 527  
03/20 00:07:22 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.04s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.15s).
Accumulating evaluation results...
DONE (t=0.90s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.303
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.655
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.223
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.230
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.357
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.556
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.393
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.334
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.440
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.600
03/20 00:07:27 - mmengine - INFO - bbox_mAP_copypaste: 0.303 0.655 0.223 0.230 0.357 0.556
03/20 00:07:27 - mmengine - INFO - Epoch(val) [92][250/250]    coco/bbox_mAP: 0.3030  coco/bbox_mAP_50: 0.6550  coco/bbox_mAP_75: 0.2230  coco/bbox_mAP_s: 0.2300  coco/bbox_mAP_m: 0.3570  coco/bbox_mAP_l: 0.5560  data_time: 0.0066  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:07:37 - mmengine - INFO - Epoch(train)  [93][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:33  time: 0.1989  data_time: 0.0098  memory: 3071  loss: 3.9465  loss_cls: 0.5597  loss_bbox: 1.9820  loss_obj: 0.7980  loss_l1: 0.6067
03/20 00:07:47 - mmengine - INFO - Epoch(train)  [93][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:22  time: 0.2042  data_time: 0.0077  memory: 3937  loss: 4.0020  loss_cls: 0.5631  loss_bbox: 1.9870  loss_obj: 0.8301  loss_l1: 0.6218
03/20 00:08:00 - mmengine - INFO - Epoch(train)  [93][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:11  time: 0.2539  data_time: 0.0078  memory: 3937  loss: 3.9824  loss_cls: 0.5516  loss_bbox: 1.9695  loss_obj: 0.7690  loss_l1: 0.6924
03/20 00:08:10 - mmengine - INFO - Epoch(train)  [93][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:28:01  time: 0.2099  data_time: 0.0077  memory: 3639  loss: 3.9902  loss_cls: 0.5595  loss_bbox: 1.9972  loss_obj: 0.7842  loss_l1: 0.6494
03/20 00:08:20 - mmengine - INFO - Epoch(train)  [93][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:27:50  time: 0.1980  data_time: 0.0078  memory: 3937  loss: 3.9967  loss_cls: 0.5601  loss_bbox: 1.9922  loss_obj: 0.8292  loss_l1: 0.6152
03/20 00:08:31 - mmengine - INFO - Epoch(train)  [93][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:27:39  time: 0.2132  data_time: 0.0077  memory: 3639  loss: 3.9901  loss_cls: 0.5512  loss_bbox: 1.9836  loss_obj: 0.8240  loss_l1: 0.6314
03/20 00:08:42 - mmengine - INFO - Epoch(train)  [93][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:27:28  time: 0.2242  data_time: 0.0077  memory: 3639  loss: 4.0275  loss_cls: 0.5565  loss_bbox: 2.0021  loss_obj: 0.8023  loss_l1: 0.6666
03/20 00:08:50 - mmengine - INFO - Epoch(train)  [93][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:27:17  time: 0.1671  data_time: 0.0078  memory: 2337  loss: 3.8988  loss_cls: 0.5629  loss_bbox: 1.9705  loss_obj: 0.8024  loss_l1: 0.5630
03/20 00:09:01 - mmengine - INFO - Epoch(train)  [93][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:27:06  time: 0.2126  data_time: 0.0077  memory: 3937  loss: 3.9169  loss_cls: 0.5575  loss_bbox: 1.9547  loss_obj: 0.7893  loss_l1: 0.6154
03/20 00:09:11 - mmengine - INFO - Epoch(train)  [93][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:56  time: 0.1991  data_time: 0.0077  memory: 3071  loss: 3.9107  loss_cls: 0.5548  loss_bbox: 1.9697  loss_obj: 0.7732  loss_l1: 0.6130
03/20 00:09:21 - mmengine - INFO - Epoch(train)  [93][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:45  time: 0.2012  data_time: 0.0077  memory: 3639  loss: 3.9463  loss_cls: 0.5556  loss_bbox: 1.9817  loss_obj: 0.7870  loss_l1: 0.6220
03/20 00:09:31 - mmengine - INFO - Epoch(train)  [93][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:34  time: 0.2071  data_time: 0.0077  memory: 3639  loss: 4.0294  loss_cls: 0.5635  loss_bbox: 2.0056  loss_obj: 0.8049  loss_l1: 0.6554
03/20 00:09:42 - mmengine - INFO - Epoch(train)  [93][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:23  time: 0.2022  data_time: 0.0076  memory: 3639  loss: 4.1008  loss_cls: 0.5721  loss_bbox: 2.0439  loss_obj: 0.8406  loss_l1: 0.6442
03/20 00:09:52 - mmengine - INFO - Epoch(train)  [93][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:12  time: 0.2042  data_time: 0.0077  memory: 3937  loss: 3.9251  loss_cls: 0.5579  loss_bbox: 1.9690  loss_obj: 0.7869  loss_l1: 0.6113
03/20 00:10:01 - mmengine - INFO - Epoch(train)  [93][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:26:01  time: 0.1897  data_time: 0.0076  memory: 3071  loss: 3.9789  loss_cls: 0.5700  loss_bbox: 1.9968  loss_obj: 0.8134  loss_l1: 0.5986
03/20 00:10:12 - mmengine - INFO - Epoch(train)  [93][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:25:51  time: 0.2181  data_time: 0.0077  memory: 3639  loss: 4.0919  loss_cls: 0.5680  loss_bbox: 2.0246  loss_obj: 0.8323  loss_l1: 0.6669
03/20 00:10:22 - mmengine - INFO - Epoch(train)  [93][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:25:40  time: 0.1870  data_time: 0.0078  memory: 2587  loss: 3.9344  loss_cls: 0.5607  loss_bbox: 1.9740  loss_obj: 0.8084  loss_l1: 0.5913
03/20 00:10:31 - mmengine - INFO - Epoch(train)  [93][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:25:29  time: 0.1969  data_time: 0.0077  memory: 3639  loss: 4.0080  loss_cls: 0.5680  loss_bbox: 1.9967  loss_obj: 0.8375  loss_l1: 0.6059
03/20 00:10:41 - mmengine - INFO - Epoch(train)  [93][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:25:18  time: 0.1876  data_time: 0.0077  memory: 2817  loss: 3.9525  loss_cls: 0.5677  loss_bbox: 1.9844  loss_obj: 0.8095  loss_l1: 0.5908
03/20 00:10:51 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:10:51 - mmengine - INFO - Epoch(train)  [93][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:25:07  time: 0.1964  data_time: 0.0076  memory: 3937  loss: 4.0236  loss_cls: 0.5717  loss_bbox: 2.0120  loss_obj: 0.8190  loss_l1: 0.6210
03/20 00:10:51 - mmengine - INFO - Saving checkpoint at 93 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:10:56 - mmengine - INFO - Epoch(val)  [93][ 50/250]    eta: 0:00:11  time: 0.0593  data_time: 0.0073  memory: 527  
03/20 00:10:59 - mmengine - INFO - Epoch(val)  [93][100/250]    eta: 0:00:08  time: 0.0585  data_time: 0.0065  memory: 527  
03/20 00:11:02 - mmengine - INFO - Epoch(val)  [93][150/250]    eta: 0:00:05  time: 0.0587  data_time: 0.0065  memory: 527  
03/20 00:11:05 - mmengine - INFO - Epoch(val)  [93][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/20 00:11:08 - mmengine - INFO - Epoch(val)  [93][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0064  memory: 527  
03/20 00:11:08 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.04s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.07s).
Accumulating evaluation results...
DONE (t=0.87s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.307
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.659
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.230
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.235
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.360
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.573
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.397
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.339
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.440
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.626
03/20 00:11:13 - mmengine - INFO - bbox_mAP_copypaste: 0.307 0.659 0.230 0.235 0.360 0.573
03/20 00:11:13 - mmengine - INFO - Epoch(val) [93][250/250]    coco/bbox_mAP: 0.3070  coco/bbox_mAP_50: 0.6590  coco/bbox_mAP_75: 0.2300  coco/bbox_mAP_s: 0.2350  coco/bbox_mAP_m: 0.3600  coco/bbox_mAP_l: 0.5730  data_time: 0.0066  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:11:23 - mmengine - INFO - Epoch(train)  [94][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:56  time: 0.1991  data_time: 0.0097  memory: 3357  loss: 3.9176  loss_cls: 0.5593  loss_bbox: 1.9790  loss_obj: 0.7676  loss_l1: 0.6117
03/20 00:11:34 - mmengine - INFO - Epoch(train)  [94][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:46  time: 0.2152  data_time: 0.0076  memory: 3937  loss: 3.9847  loss_cls: 0.5603  loss_bbox: 1.9945  loss_obj: 0.8001  loss_l1: 0.6299
03/20 00:11:45 - mmengine - INFO - Epoch(train)  [94][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:35  time: 0.2117  data_time: 0.0077  memory: 3639  loss: 3.8511  loss_cls: 0.5526  loss_bbox: 1.9374  loss_obj: 0.7399  loss_l1: 0.6212
03/20 00:11:55 - mmengine - INFO - Epoch(train)  [94][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:24  time: 0.2105  data_time: 0.0076  memory: 3937  loss: 4.0925  loss_cls: 0.5677  loss_bbox: 2.0381  loss_obj: 0.8344  loss_l1: 0.6523
03/20 00:12:05 - mmengine - INFO - Epoch(train)  [94][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:13  time: 0.1906  data_time: 0.0077  memory: 3071  loss: 3.9699  loss_cls: 0.5663  loss_bbox: 1.9855  loss_obj: 0.8130  loss_l1: 0.6050
03/20 00:12:14 - mmengine - INFO - Epoch(train)  [94][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:24:02  time: 0.1922  data_time: 0.0077  memory: 3357  loss: 3.9134  loss_cls: 0.5528  loss_bbox: 1.9609  loss_obj: 0.7969  loss_l1: 0.6027
03/20 00:12:23 - mmengine - INFO - Epoch(train)  [94][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:23:51  time: 0.1800  data_time: 0.0077  memory: 3071  loss: 3.9304  loss_cls: 0.5642  loss_bbox: 1.9794  loss_obj: 0.7929  loss_l1: 0.5939
03/20 00:12:35 - mmengine - INFO - Epoch(train)  [94][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:23:41  time: 0.2325  data_time: 0.0078  memory: 3937  loss: 3.9951  loss_cls: 0.5487  loss_bbox: 1.9698  loss_obj: 0.8060  loss_l1: 0.6707
03/20 00:12:46 - mmengine - INFO - Epoch(train)  [94][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:23:30  time: 0.2187  data_time: 0.0078  memory: 3639  loss: 3.9414  loss_cls: 0.5491  loss_bbox: 1.9654  loss_obj: 0.7967  loss_l1: 0.6301
03/20 00:12:57 - mmengine - INFO - Epoch(train)  [94][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:23:19  time: 0.2235  data_time: 0.0077  memory: 3937  loss: 3.9048  loss_cls: 0.5563  loss_bbox: 1.9549  loss_obj: 0.7554  loss_l1: 0.6382
03/20 00:13:07 - mmengine - INFO - Epoch(train)  [94][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:23:08  time: 0.1907  data_time: 0.0077  memory: 2817  loss: 3.9785  loss_cls: 0.5594  loss_bbox: 1.9949  loss_obj: 0.8090  loss_l1: 0.6152
03/20 00:13:17 - mmengine - INFO - Epoch(train)  [94][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:58  time: 0.2110  data_time: 0.0078  memory: 3639  loss: 3.9826  loss_cls: 0.5646  loss_bbox: 1.9968  loss_obj: 0.7789  loss_l1: 0.6423
03/20 00:13:27 - mmengine - INFO - Epoch(train)  [94][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:47  time: 0.1954  data_time: 0.0077  memory: 3937  loss: 3.8994  loss_cls: 0.5548  loss_bbox: 1.9490  loss_obj: 0.7935  loss_l1: 0.6021
03/20 00:13:38 - mmengine - INFO - Epoch(train)  [94][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:36  time: 0.2246  data_time: 0.0077  memory: 3937  loss: 4.0113  loss_cls: 0.5560  loss_bbox: 1.9907  loss_obj: 0.8207  loss_l1: 0.6440
03/20 00:13:49 - mmengine - INFO - Epoch(train)  [94][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:25  time: 0.2131  data_time: 0.0078  memory: 3071  loss: 3.9409  loss_cls: 0.5587  loss_bbox: 1.9816  loss_obj: 0.7671  loss_l1: 0.6335
03/20 00:13:59 - mmengine - INFO - Epoch(train)  [94][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:14  time: 0.1991  data_time: 0.0078  memory: 3639  loss: 4.0360  loss_cls: 0.5690  loss_bbox: 2.0333  loss_obj: 0.8047  loss_l1: 0.6290
03/20 00:14:09 - mmengine - INFO - Epoch(train)  [94][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:22:04  time: 0.2079  data_time: 0.0077  memory: 3357  loss: 4.0347  loss_cls: 0.5615  loss_bbox: 2.0012  loss_obj: 0.8341  loss_l1: 0.6380
03/20 00:14:21 - mmengine - INFO - Epoch(train)  [94][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:21:53  time: 0.2325  data_time: 0.0077  memory: 3937  loss: 4.0497  loss_cls: 0.5625  loss_bbox: 2.0018  loss_obj: 0.8167  loss_l1: 0.6687
03/20 00:14:31 - mmengine - INFO - Epoch(train)  [94][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:21:42  time: 0.2026  data_time: 0.0077  memory: 3937  loss: 3.9579  loss_cls: 0.5584  loss_bbox: 1.9746  loss_obj: 0.8126  loss_l1: 0.6124
03/20 00:14:41 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:14:41 - mmengine - INFO - Epoch(train)  [94][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:21:31  time: 0.2059  data_time: 0.0076  memory: 3357  loss: 3.9544  loss_cls: 0.5580  loss_bbox: 1.9601  loss_obj: 0.8252  loss_l1: 0.6110
03/20 00:14:41 - mmengine - INFO - Saving checkpoint at 94 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:14:47 - mmengine - INFO - Epoch(val)  [94][ 50/250]    eta: 0:00:11  time: 0.0586  data_time: 0.0071  memory: 527  
03/20 00:14:50 - mmengine - INFO - Epoch(val)  [94][100/250]    eta: 0:00:08  time: 0.0594  data_time: 0.0065  memory: 527  
03/20 00:14:52 - mmengine - INFO - Epoch(val)  [94][150/250]    eta: 0:00:05  time: 0.0577  data_time: 0.0064  memory: 527  
03/20 00:14:55 - mmengine - INFO - Epoch(val)  [94][200/250]    eta: 0:00:02  time: 0.0573  data_time: 0.0064  memory: 527  
03/20 00:14:58 - mmengine - INFO - Epoch(val)  [94][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0065  memory: 527  
03/20 00:14:59 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.04s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.97s).
Accumulating evaluation results...
DONE (t=0.85s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.309
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.658
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.234
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.237
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.366
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.571
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.339
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.444
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.625
03/20 00:15:04 - mmengine - INFO - bbox_mAP_copypaste: 0.309 0.658 0.234 0.237 0.366 0.571
03/20 00:15:04 - mmengine - INFO - Epoch(val) [94][250/250]    coco/bbox_mAP: 0.3090  coco/bbox_mAP_50: 0.6580  coco/bbox_mAP_75: 0.2340  coco/bbox_mAP_s: 0.2370  coco/bbox_mAP_m: 0.3660  coco/bbox_mAP_l: 0.5710  data_time: 0.0066  time: 0.0580
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:15:14 - mmengine - INFO - Epoch(train)  [95][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:21:21  time: 0.1981  data_time: 0.0099  memory: 3357  loss: 3.8578  loss_cls: 0.5618  loss_bbox: 1.9659  loss_obj: 0.7389  loss_l1: 0.5912
03/20 00:15:24 - mmengine - INFO - Epoch(train)  [95][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:21:10  time: 0.2096  data_time: 0.0077  memory: 3937  loss: 3.9067  loss_cls: 0.5577  loss_bbox: 1.9583  loss_obj: 0.7797  loss_l1: 0.6110
03/20 00:15:35 - mmengine - INFO - Epoch(train)  [95][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:59  time: 0.2080  data_time: 0.0077  memory: 3639  loss: 3.8461  loss_cls: 0.5475  loss_bbox: 1.9373  loss_obj: 0.7469  loss_l1: 0.6144
03/20 00:15:47 - mmengine - INFO - Epoch(train)  [95][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:48  time: 0.2385  data_time: 0.0078  memory: 3937  loss: 3.8629  loss_cls: 0.5401  loss_bbox: 1.9313  loss_obj: 0.7502  loss_l1: 0.6413
03/20 00:15:56 - mmengine - INFO - Epoch(train)  [95][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:37  time: 0.1811  data_time: 0.0077  memory: 2587  loss: 3.8968  loss_cls: 0.5617  loss_bbox: 1.9638  loss_obj: 0.8012  loss_l1: 0.5700
03/20 00:16:07 - mmengine - INFO - Epoch(train)  [95][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:27  time: 0.2218  data_time: 0.0077  memory: 3937  loss: 3.9808  loss_cls: 0.5564  loss_bbox: 1.9877  loss_obj: 0.7902  loss_l1: 0.6465
03/20 00:16:17 - mmengine - INFO - Epoch(train)  [95][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:16  time: 0.1979  data_time: 0.0078  memory: 3639  loss: 3.9356  loss_cls: 0.5681  loss_bbox: 1.9807  loss_obj: 0.7757  loss_l1: 0.6112
03/20 00:16:27 - mmengine - INFO - Epoch(train)  [95][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:20:05  time: 0.2073  data_time: 0.0078  memory: 3937  loss: 4.0253  loss_cls: 0.5620  loss_bbox: 2.0119  loss_obj: 0.8058  loss_l1: 0.6456
03/20 00:16:37 - mmengine - INFO - Epoch(train)  [95][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:54  time: 0.2074  data_time: 0.0077  memory: 3357  loss: 3.9352  loss_cls: 0.5459  loss_bbox: 1.9687  loss_obj: 0.7983  loss_l1: 0.6223
03/20 00:16:48 - mmengine - INFO - Epoch(train)  [95][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:44  time: 0.2076  data_time: 0.0077  memory: 3639  loss: 3.9223  loss_cls: 0.5549  loss_bbox: 1.9645  loss_obj: 0.7811  loss_l1: 0.6218
03/20 00:16:58 - mmengine - INFO - Epoch(train)  [95][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:33  time: 0.2080  data_time: 0.0077  memory: 3639  loss: 4.0508  loss_cls: 0.5711  loss_bbox: 2.0240  loss_obj: 0.8191  loss_l1: 0.6366
03/20 00:17:08 - mmengine - INFO - Epoch(train)  [95][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:22  time: 0.1928  data_time: 0.0077  memory: 2817  loss: 3.9618  loss_cls: 0.5708  loss_bbox: 1.9902  loss_obj: 0.8029  loss_l1: 0.5979
03/20 00:17:18 - mmengine - INFO - Epoch(train)  [95][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:11  time: 0.1979  data_time: 0.0077  memory: 3357  loss: 3.9231  loss_cls: 0.5586  loss_bbox: 1.9709  loss_obj: 0.7798  loss_l1: 0.6138
03/20 00:17:28 - mmengine - INFO - Epoch(train)  [95][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:19:00  time: 0.2114  data_time: 0.0077  memory: 3639  loss: 3.9052  loss_cls: 0.5497  loss_bbox: 1.9501  loss_obj: 0.7897  loss_l1: 0.6157
03/20 00:17:38 - mmengine - INFO - Epoch(train)  [95][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:18:50  time: 0.2000  data_time: 0.0077  memory: 3937  loss: 3.9660  loss_cls: 0.5634  loss_bbox: 1.9851  loss_obj: 0.7909  loss_l1: 0.6266
03/20 00:17:48 - mmengine - INFO - Epoch(train)  [95][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:18:39  time: 0.1987  data_time: 0.0077  memory: 3357  loss: 3.9410  loss_cls: 0.5549  loss_bbox: 1.9732  loss_obj: 0.7893  loss_l1: 0.6236
03/20 00:17:59 - mmengine - INFO - Epoch(train)  [95][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:18:28  time: 0.2071  data_time: 0.0077  memory: 3639  loss: 3.8908  loss_cls: 0.5446  loss_bbox: 1.9531  loss_obj: 0.7799  loss_l1: 0.6132
03/20 00:18:10 - mmengine - INFO - Epoch(train)  [95][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:18:17  time: 0.2288  data_time: 0.0077  memory: 3937  loss: 4.0066  loss_cls: 0.5524  loss_bbox: 1.9757  loss_obj: 0.8222  loss_l1: 0.6562
03/20 00:18:21 - mmengine - INFO - Epoch(train)  [95][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:18:06  time: 0.2254  data_time: 0.0078  memory: 3937  loss: 4.0468  loss_cls: 0.5676  loss_bbox: 2.0078  loss_obj: 0.8205  loss_l1: 0.6508
03/20 00:18:31 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:18:31 - mmengine - INFO - Epoch(train)  [95][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:56  time: 0.1969  data_time: 0.0076  memory: 3071  loss: 3.9442  loss_cls: 0.5535  loss_bbox: 1.9678  loss_obj: 0.8135  loss_l1: 0.6094
03/20 00:18:31 - mmengine - INFO - Saving checkpoint at 95 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:18:37 - mmengine - INFO - Epoch(val)  [95][ 50/250]    eta: 0:00:11  time: 0.0589  data_time: 0.0073  memory: 527  
03/20 00:18:39 - mmengine - INFO - Epoch(val)  [95][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0064  memory: 527  
03/20 00:18:42 - mmengine - INFO - Epoch(val)  [95][150/250]    eta: 0:00:05  time: 0.0582  data_time: 0.0064  memory: 527  
03/20 00:18:45 - mmengine - INFO - Epoch(val)  [95][200/250]    eta: 0:00:02  time: 0.0585  data_time: 0.0066  memory: 527  
03/20 00:18:48 - mmengine - INFO - Epoch(val)  [95][250/250]    eta: 0:00:00  time: 0.0565  data_time: 0.0065  memory: 527  
03/20 00:18:49 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.03s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.93s).
Accumulating evaluation results...
DONE (t=0.84s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.316
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.663
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.238
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.245
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.376
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.578
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.407
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.407
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.407
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.347
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.452
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.629
03/20 00:18:54 - mmengine - INFO - bbox_mAP_copypaste: 0.316 0.663 0.238 0.245 0.376 0.578
03/20 00:18:54 - mmengine - INFO - Epoch(val) [95][250/250]    coco/bbox_mAP: 0.3160  coco/bbox_mAP_50: 0.6630  coco/bbox_mAP_75: 0.2380  coco/bbox_mAP_s: 0.2450  coco/bbox_mAP_m: 0.3760  coco/bbox_mAP_l: 0.5780  data_time: 0.0066  time: 0.0580
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:19:03 - mmengine - INFO - Epoch(train)  [96][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:45  time: 0.1969  data_time: 0.0098  memory: 3071  loss: 3.9017  loss_cls: 0.5575  loss_bbox: 1.9730  loss_obj: 0.7733  loss_l1: 0.5980
03/20 00:19:14 - mmengine - INFO - Epoch(train)  [96][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:34  time: 0.2036  data_time: 0.0077  memory: 3639  loss: 3.7932  loss_cls: 0.5424  loss_bbox: 1.9160  loss_obj: 0.7333  loss_l1: 0.6015
03/20 00:19:26 - mmengine - INFO - Epoch(train)  [96][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:23  time: 0.2363  data_time: 0.0077  memory: 3937  loss: 3.9593  loss_cls: 0.5641  loss_bbox: 1.9665  loss_obj: 0.7708  loss_l1: 0.6579
03/20 00:19:34 - mmengine - INFO - Epoch(train)  [96][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:12  time: 0.1749  data_time: 0.0078  memory: 2587  loss: 3.9611  loss_cls: 0.5718  loss_bbox: 1.9837  loss_obj: 0.8200  loss_l1: 0.5856
03/20 00:19:45 - mmengine - INFO - Epoch(train)  [96][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:17:02  time: 0.2073  data_time: 0.0077  memory: 3357  loss: 3.9791  loss_cls: 0.5725  loss_bbox: 1.9937  loss_obj: 0.7834  loss_l1: 0.6295
03/20 00:19:55 - mmengine - INFO - Epoch(train)  [96][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:16:51  time: 0.2141  data_time: 0.0078  memory: 3639  loss: 3.8719  loss_cls: 0.5483  loss_bbox: 1.9567  loss_obj: 0.7399  loss_l1: 0.6270
03/20 00:20:05 - mmengine - INFO - Epoch(train)  [96][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:16:40  time: 0.1920  data_time: 0.0078  memory: 3071  loss: 3.8660  loss_cls: 0.5537  loss_bbox: 1.9414  loss_obj: 0.7848  loss_l1: 0.5860
03/20 00:20:16 - mmengine - INFO - Epoch(train)  [96][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:16:29  time: 0.2139  data_time: 0.0078  memory: 3639  loss: 3.8140  loss_cls: 0.5376  loss_bbox: 1.9163  loss_obj: 0.7447  loss_l1: 0.6154
03/20 00:20:26 - mmengine - INFO - Epoch(train)  [96][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:16:19  time: 0.2031  data_time: 0.0078  memory: 3639  loss: 3.9236  loss_cls: 0.5504  loss_bbox: 1.9541  loss_obj: 0.8134  loss_l1: 0.6058
03/20 00:20:36 - mmengine - INFO - Epoch(train)  [96][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:16:08  time: 0.2108  data_time: 0.0077  memory: 3357  loss: 3.7983  loss_cls: 0.5429  loss_bbox: 1.9154  loss_obj: 0.7386  loss_l1: 0.6014
03/20 00:20:46 - mmengine - INFO - Epoch(train)  [96][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:57  time: 0.1942  data_time: 0.0078  memory: 3357  loss: 3.8791  loss_cls: 0.5563  loss_bbox: 1.9495  loss_obj: 0.7781  loss_l1: 0.5953
03/20 00:20:58 - mmengine - INFO - Epoch(train)  [96][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:46  time: 0.2439  data_time: 0.0077  memory: 3937  loss: 3.9594  loss_cls: 0.5483  loss_bbox: 1.9627  loss_obj: 0.7721  loss_l1: 0.6763
03/20 00:21:08 - mmengine - INFO - Epoch(train)  [96][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:36  time: 0.2030  data_time: 0.0077  memory: 3639  loss: 3.9411  loss_cls: 0.5559  loss_bbox: 1.9641  loss_obj: 0.7994  loss_l1: 0.6218
03/20 00:21:18 - mmengine - INFO - Epoch(train)  [96][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:25  time: 0.1812  data_time: 0.0077  memory: 3357  loss: 3.9891  loss_cls: 0.5687  loss_bbox: 1.9948  loss_obj: 0.8401  loss_l1: 0.5856
03/20 00:21:27 - mmengine - INFO - Epoch(train)  [96][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:14  time: 0.1917  data_time: 0.0077  memory: 3071  loss: 3.9448  loss_cls: 0.5585  loss_bbox: 1.9910  loss_obj: 0.7831  loss_l1: 0.6123
03/20 00:21:37 - mmengine - INFO - Epoch(train)  [96][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:15:03  time: 0.2001  data_time: 0.0077  memory: 3357  loss: 3.9228  loss_cls: 0.5548  loss_bbox: 1.9714  loss_obj: 0.7858  loss_l1: 0.6107
03/20 00:21:47 - mmengine - INFO - Epoch(train)  [96][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:14:52  time: 0.1910  data_time: 0.0077  memory: 3071  loss: 3.8925  loss_cls: 0.5570  loss_bbox: 1.9747  loss_obj: 0.7619  loss_l1: 0.5989
03/20 00:21:57 - mmengine - INFO - Epoch(train)  [96][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:14:41  time: 0.2008  data_time: 0.0076  memory: 3639  loss: 3.8616  loss_cls: 0.5515  loss_bbox: 1.9429  loss_obj: 0.7619  loss_l1: 0.6052
03/20 00:22:07 - mmengine - INFO - Epoch(train)  [96][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:14:31  time: 0.2034  data_time: 0.0077  memory: 3937  loss: 4.0134  loss_cls: 0.5621  loss_bbox: 2.0262  loss_obj: 0.7945  loss_l1: 0.6306
03/20 00:22:18 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:22:18 - mmengine - INFO - Epoch(train)  [96][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:14:20  time: 0.2218  data_time: 0.0077  memory: 3937  loss: 3.9648  loss_cls: 0.5574  loss_bbox: 1.9677  loss_obj: 0.8090  loss_l1: 0.6308
03/20 00:22:18 - mmengine - INFO - Saving checkpoint at 96 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:22:23 - mmengine - INFO - Epoch(val)  [96][ 50/250]    eta: 0:00:11  time: 0.0585  data_time: 0.0072  memory: 527  
03/20 00:22:26 - mmengine - INFO - Epoch(val)  [96][100/250]    eta: 0:00:08  time: 0.0589  data_time: 0.0066  memory: 527  
03/20 00:22:29 - mmengine - INFO - Epoch(val)  [96][150/250]    eta: 0:00:05  time: 0.0588  data_time: 0.0065  memory: 527  
03/20 00:22:32 - mmengine - INFO - Epoch(val)  [96][200/250]    eta: 0:00:02  time: 0.0589  data_time: 0.0065  memory: 527  
03/20 00:22:35 - mmengine - INFO - Epoch(val)  [96][250/250]    eta: 0:00:00  time: 0.0569  data_time: 0.0065  memory: 527  
03/20 00:22:36 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.03s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.88s).
Accumulating evaluation results...
DONE (t=0.82s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.319
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.669
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.246
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.247
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.578
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.410
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.410
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.410
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.350
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.454
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.630
03/20 00:22:41 - mmengine - INFO - bbox_mAP_copypaste: 0.319 0.669 0.246 0.247 0.381 0.578
03/20 00:22:41 - mmengine - INFO - Epoch(val) [96][250/250]    coco/bbox_mAP: 0.3190  coco/bbox_mAP_50: 0.6690  coco/bbox_mAP_75: 0.2460  coco/bbox_mAP_s: 0.2470  coco/bbox_mAP_m: 0.3810  coco/bbox_mAP_l: 0.5780  data_time: 0.0067  time: 0.0584
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:22:51 - mmengine - INFO - Epoch(train)  [97][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:14:09  time: 0.1986  data_time: 0.0097  memory: 2817  loss: 3.7356  loss_cls: 0.5380  loss_bbox: 1.9033  loss_obj: 0.7167  loss_l1: 0.5776
03/20 00:23:02 - mmengine - INFO - Epoch(train)  [97][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:58  time: 0.2202  data_time: 0.0078  memory: 3639  loss: 3.8814  loss_cls: 0.5444  loss_bbox: 1.9545  loss_obj: 0.7522  loss_l1: 0.6304
03/20 00:23:12 - mmengine - INFO - Epoch(train)  [97][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:48  time: 0.2076  data_time: 0.0077  memory: 3937  loss: 3.9192  loss_cls: 0.5511  loss_bbox: 1.9651  loss_obj: 0.7910  loss_l1: 0.6121
03/20 00:23:23 - mmengine - INFO - Epoch(train)  [97][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:37  time: 0.2119  data_time: 0.0077  memory: 3937  loss: 3.9119  loss_cls: 0.5522  loss_bbox: 1.9679  loss_obj: 0.7780  loss_l1: 0.6138
03/20 00:23:33 - mmengine - INFO - Epoch(train)  [97][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:26  time: 0.2059  data_time: 0.0077  memory: 3937  loss: 3.9156  loss_cls: 0.5544  loss_bbox: 1.9655  loss_obj: 0.7895  loss_l1: 0.6061
03/20 00:23:42 - mmengine - INFO - Epoch(train)  [97][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:15  time: 0.1773  data_time: 0.0077  memory: 2337  loss: 3.8453  loss_cls: 0.5496  loss_bbox: 1.9644  loss_obj: 0.7677  loss_l1: 0.5636
03/20 00:23:51 - mmengine - INFO - Epoch(train)  [97][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:13:04  time: 0.1810  data_time: 0.0078  memory: 3357  loss: 3.8539  loss_cls: 0.5495  loss_bbox: 1.9333  loss_obj: 0.8034  loss_l1: 0.5677
03/20 00:24:01 - mmengine - INFO - Epoch(train)  [97][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:54  time: 0.2058  data_time: 0.0077  memory: 3639  loss: 3.8473  loss_cls: 0.5526  loss_bbox: 1.9404  loss_obj: 0.7529  loss_l1: 0.6015
03/20 00:24:11 - mmengine - INFO - Epoch(train)  [97][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:43  time: 0.1923  data_time: 0.0077  memory: 3937  loss: 3.9514  loss_cls: 0.5662  loss_bbox: 1.9759  loss_obj: 0.8190  loss_l1: 0.5902
03/20 00:24:22 - mmengine - INFO - Epoch(train)  [97][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:32  time: 0.2196  data_time: 0.0077  memory: 3937  loss: 3.9579  loss_cls: 0.5554  loss_bbox: 1.9669  loss_obj: 0.7981  loss_l1: 0.6374
03/20 00:24:32 - mmengine - INFO - Epoch(train)  [97][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:21  time: 0.2050  data_time: 0.0077  memory: 3639  loss: 3.8873  loss_cls: 0.5541  loss_bbox: 1.9571  loss_obj: 0.7649  loss_l1: 0.6112
03/20 00:24:42 - mmengine - INFO - Epoch(train)  [97][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:11  time: 0.1918  data_time: 0.0077  memory: 3639  loss: 3.8988  loss_cls: 0.5593  loss_bbox: 1.9601  loss_obj: 0.7859  loss_l1: 0.5935
03/20 00:24:52 - mmengine - INFO - Epoch(train)  [97][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:12:00  time: 0.2046  data_time: 0.0076  memory: 3071  loss: 3.9278  loss_cls: 0.5646  loss_bbox: 1.9587  loss_obj: 0.7866  loss_l1: 0.6179
03/20 00:25:02 - mmengine - INFO - Epoch(train)  [97][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:11:49  time: 0.1988  data_time: 0.0076  memory: 3937  loss: 3.9185  loss_cls: 0.5579  loss_bbox: 1.9688  loss_obj: 0.7858  loss_l1: 0.6060
03/20 00:25:10 - mmengine - INFO - Epoch(train)  [97][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:11:38  time: 0.1677  data_time: 0.0077  memory: 2587  loss: 3.9256  loss_cls: 0.5725  loss_bbox: 1.9853  loss_obj: 0.7985  loss_l1: 0.5693
03/20 00:25:21 - mmengine - INFO - Epoch(train)  [97][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:11:27  time: 0.2085  data_time: 0.0077  memory: 3639  loss: 3.8557  loss_cls: 0.5525  loss_bbox: 1.9319  loss_obj: 0.7754  loss_l1: 0.5959
03/20 00:25:32 - mmengine - INFO - Epoch(train)  [97][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:11:17  time: 0.2185  data_time: 0.0078  memory: 3937  loss: 4.0010  loss_cls: 0.5635  loss_bbox: 1.9912  loss_obj: 0.8060  loss_l1: 0.6403
03/20 00:25:40 - mmengine - INFO - Epoch(train)  [97][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:11:06  time: 0.1768  data_time: 0.0077  memory: 2337  loss: 3.8198  loss_cls: 0.5510  loss_bbox: 1.9364  loss_obj: 0.7717  loss_l1: 0.5606
03/20 00:25:50 - mmengine - INFO - Epoch(train)  [97][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:55  time: 0.1965  data_time: 0.0077  memory: 3071  loss: 3.8663  loss_cls: 0.5510  loss_bbox: 1.9498  loss_obj: 0.7735  loss_l1: 0.5919
03/20 00:26:02 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:26:02 - mmengine - INFO - Epoch(train)  [97][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:44  time: 0.2261  data_time: 0.0078  memory: 3937  loss: 3.8212  loss_cls: 0.5429  loss_bbox: 1.9181  loss_obj: 0.7375  loss_l1: 0.6226
03/20 00:26:02 - mmengine - INFO - Saving checkpoint at 97 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:26:07 - mmengine - INFO - Epoch(val)  [97][ 50/250]    eta: 0:00:12  time: 0.0601  data_time: 0.0073  memory: 527  
03/20 00:26:10 - mmengine - INFO - Epoch(val)  [97][100/250]    eta: 0:00:08  time: 0.0582  data_time: 0.0065  memory: 527  
03/20 00:26:13 - mmengine - INFO - Epoch(val)  [97][150/250]    eta: 0:00:05  time: 0.0583  data_time: 0.0065  memory: 527  
03/20 00:26:16 - mmengine - INFO - Epoch(val)  [97][200/250]    eta: 0:00:02  time: 0.0584  data_time: 0.0065  memory: 527  
03/20 00:26:19 - mmengine - INFO - Epoch(val)  [97][250/250]    eta: 0:00:00  time: 0.0563  data_time: 0.0065  memory: 527  
03/20 00:26:19 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.03s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.87s).
Accumulating evaluation results...
DONE (t=0.82s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.322
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.673
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.251
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.249
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.385
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.565
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.412
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.412
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.412
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.352
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.456
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.636
03/20 00:26:24 - mmengine - INFO - bbox_mAP_copypaste: 0.322 0.673 0.251 0.249 0.385 0.565
03/20 00:26:24 - mmengine - INFO - Epoch(val) [97][250/250]    coco/bbox_mAP: 0.3220  coco/bbox_mAP_50: 0.6730  coco/bbox_mAP_75: 0.2510  coco/bbox_mAP_s: 0.2490  coco/bbox_mAP_m: 0.3850  coco/bbox_mAP_l: 0.5650  data_time: 0.0067  time: 0.0583
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:26:33 - mmengine - INFO - Epoch(train)  [98][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:34  time: 0.1742  data_time: 0.0100  memory: 2817  loss: 3.8277  loss_cls: 0.5573  loss_bbox: 1.9465  loss_obj: 0.7676  loss_l1: 0.5562
03/20 00:26:43 - mmengine - INFO - Epoch(train)  [98][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:23  time: 0.2119  data_time: 0.0078  memory: 3937  loss: 3.8454  loss_cls: 0.5425  loss_bbox: 1.9467  loss_obj: 0.7435  loss_l1: 0.6128
03/20 00:26:52 - mmengine - INFO - Epoch(train)  [98][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:12  time: 0.1758  data_time: 0.0076  memory: 2587  loss: 3.8722  loss_cls: 0.5654  loss_bbox: 1.9456  loss_obj: 0.8078  loss_l1: 0.5534
03/20 00:27:03 - mmengine - INFO - Epoch(train)  [98][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:10:01  time: 0.2165  data_time: 0.0077  memory: 3937  loss: 3.8556  loss_cls: 0.5475  loss_bbox: 1.9445  loss_obj: 0.7556  loss_l1: 0.6079
03/20 00:27:12 - mmengine - INFO - Epoch(train)  [98][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:09:50  time: 0.1762  data_time: 0.0078  memory: 2587  loss: 3.8747  loss_cls: 0.5629  loss_bbox: 1.9593  loss_obj: 0.7879  loss_l1: 0.5646
03/20 00:27:22 - mmengine - INFO - Epoch(train)  [98][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:09:40  time: 0.1998  data_time: 0.0077  memory: 3357  loss: 3.8870  loss_cls: 0.5601  loss_bbox: 1.9641  loss_obj: 0.7654  loss_l1: 0.5974
03/20 00:27:31 - mmengine - INFO - Epoch(train)  [98][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:09:29  time: 0.1865  data_time: 0.0077  memory: 2817  loss: 3.7681  loss_cls: 0.5484  loss_bbox: 1.9186  loss_obj: 0.7379  loss_l1: 0.5632
03/20 00:27:41 - mmengine - INFO - Epoch(train)  [98][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:09:18  time: 0.1879  data_time: 0.0077  memory: 2817  loss: 3.8280  loss_cls: 0.5525  loss_bbox: 1.9577  loss_obj: 0.7403  loss_l1: 0.5776
03/20 00:27:53 - mmengine - INFO - Epoch(train)  [98][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:09:07  time: 0.2485  data_time: 0.0077  memory: 3937  loss: 4.0201  loss_cls: 0.5542  loss_bbox: 1.9782  loss_obj: 0.8160  loss_l1: 0.6717
03/20 00:28:03 - mmengine - INFO - Epoch(train)  [98][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:57  time: 0.2078  data_time: 0.0078  memory: 3639  loss: 3.8892  loss_cls: 0.5615  loss_bbox: 1.9523  loss_obj: 0.7717  loss_l1: 0.6038
03/20 00:28:14 - mmengine - INFO - Epoch(train)  [98][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:46  time: 0.2076  data_time: 0.0077  memory: 3937  loss: 3.9254  loss_cls: 0.5672  loss_bbox: 1.9557  loss_obj: 0.7977  loss_l1: 0.6048
03/20 00:28:23 - mmengine - INFO - Epoch(train)  [98][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:35  time: 0.1741  data_time: 0.0077  memory: 3357  loss: 4.0151  loss_cls: 0.5690  loss_bbox: 2.0172  loss_obj: 0.8473  loss_l1: 0.5816
03/20 00:28:32 - mmengine - INFO - Epoch(train)  [98][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:24  time: 0.1870  data_time: 0.0077  memory: 3071  loss: 3.8220  loss_cls: 0.5444  loss_bbox: 1.9215  loss_obj: 0.7794  loss_l1: 0.5767
03/20 00:28:42 - mmengine - INFO - Epoch(train)  [98][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:14  time: 0.1961  data_time: 0.0077  memory: 3937  loss: 3.9277  loss_cls: 0.5612  loss_bbox: 1.9817  loss_obj: 0.7805  loss_l1: 0.6043
03/20 00:28:51 - mmengine - INFO - Epoch(train)  [98][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:08:03  time: 0.1830  data_time: 0.0078  memory: 3071  loss: 3.8181  loss_cls: 0.5556  loss_bbox: 1.9392  loss_obj: 0.7486  loss_l1: 0.5747
03/20 00:29:02 - mmengine - INFO - Epoch(train)  [98][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:07:52  time: 0.2186  data_time: 0.0077  memory: 3937  loss: 3.8158  loss_cls: 0.5448  loss_bbox: 1.9101  loss_obj: 0.7542  loss_l1: 0.6067
03/20 00:29:12 - mmengine - INFO - Epoch(train)  [98][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:07:41  time: 0.1973  data_time: 0.0078  memory: 3937  loss: 3.7950  loss_cls: 0.5427  loss_bbox: 1.9141  loss_obj: 0.7605  loss_l1: 0.5777
03/20 00:29:22 - mmengine - INFO - Epoch(train)  [98][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:07:31  time: 0.2025  data_time: 0.0077  memory: 3639  loss: 3.8584  loss_cls: 0.5450  loss_bbox: 1.9526  loss_obj: 0.7564  loss_l1: 0.6044
03/20 00:29:34 - mmengine - INFO - Epoch(train)  [98][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:07:20  time: 0.2349  data_time: 0.0078  memory: 3937  loss: 3.8738  loss_cls: 0.5401  loss_bbox: 1.9322  loss_obj: 0.7603  loss_l1: 0.6411
03/20 00:29:44 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:29:44 - mmengine - INFO - Epoch(train)  [98][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:07:09  time: 0.2146  data_time: 0.0077  memory: 3639  loss: 3.7853  loss_cls: 0.5421  loss_bbox: 1.8972  loss_obj: 0.7516  loss_l1: 0.5943
03/20 00:29:44 - mmengine - INFO - Saving checkpoint at 98 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:29:50 - mmengine - INFO - Epoch(val)  [98][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0071  memory: 527  
03/20 00:29:53 - mmengine - INFO - Epoch(val)  [98][100/250]    eta: 0:00:08  time: 0.0575  data_time: 0.0065  memory: 527  
03/20 00:29:56 - mmengine - INFO - Epoch(val)  [98][150/250]    eta: 0:00:05  time: 0.0580  data_time: 0.0065  memory: 527  
03/20 00:29:58 - mmengine - INFO - Epoch(val)  [98][200/250]    eta: 0:00:02  time: 0.0581  data_time: 0.0065  memory: 527  
03/20 00:30:01 - mmengine - INFO - Epoch(val)  [98][250/250]    eta: 0:00:00  time: 0.0571  data_time: 0.0066  memory: 527  
03/20 00:30:02 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.17s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.83s).
Accumulating evaluation results...
DONE (t=0.82s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.325
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.676
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.253
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.253
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.386
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.560
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.416
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.416
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.416
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.358
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.458
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.632
03/20 00:30:07 - mmengine - INFO - bbox_mAP_copypaste: 0.325 0.676 0.253 0.253 0.386 0.560
03/20 00:30:07 - mmengine - INFO - Epoch(val) [98][250/250]    coco/bbox_mAP: 0.3250  coco/bbox_mAP_50: 0.6760  coco/bbox_mAP_75: 0.2530  coco/bbox_mAP_s: 0.2530  coco/bbox_mAP_m: 0.3860  coco/bbox_mAP_l: 0.5600  data_time: 0.0067  time: 0.0580
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:30:16 - mmengine - INFO - Epoch(train)  [99][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:58  time: 0.1889  data_time: 0.0098  memory: 3357  loss: 3.8653  loss_cls: 0.5502  loss_bbox: 1.9566  loss_obj: 0.7753  loss_l1: 0.5832
03/20 00:30:26 - mmengine - INFO - Epoch(train)  [99][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:48  time: 0.2035  data_time: 0.0077  memory: 3357  loss: 3.7745  loss_cls: 0.5383  loss_bbox: 1.9105  loss_obj: 0.7349  loss_l1: 0.5908
03/20 00:30:36 - mmengine - INFO - Epoch(train)  [99][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:37  time: 0.1988  data_time: 0.0077  memory: 3357  loss: 3.7568  loss_cls: 0.5396  loss_bbox: 1.9133  loss_obj: 0.7279  loss_l1: 0.5760
03/20 00:30:47 - mmengine - INFO - Epoch(train)  [99][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:26  time: 0.2233  data_time: 0.0077  memory: 3937  loss: 3.9487  loss_cls: 0.5586  loss_bbox: 1.9787  loss_obj: 0.7837  loss_l1: 0.6277
03/20 00:30:57 - mmengine - INFO - Epoch(train)  [99][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:15  time: 0.1851  data_time: 0.0076  memory: 3071  loss: 3.8711  loss_cls: 0.5561  loss_bbox: 1.9519  loss_obj: 0.7914  loss_l1: 0.5717
03/20 00:31:08 - mmengine - INFO - Epoch(train)  [99][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:06:05  time: 0.2239  data_time: 0.0078  memory: 3937  loss: 3.8832  loss_cls: 0.5507  loss_bbox: 1.9423  loss_obj: 0.7708  loss_l1: 0.6194
03/20 00:31:18 - mmengine - INFO - Epoch(train)  [99][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:54  time: 0.1918  data_time: 0.0077  memory: 2817  loss: 3.8070  loss_cls: 0.5459  loss_bbox: 1.9293  loss_obj: 0.7567  loss_l1: 0.5751
03/20 00:31:29 - mmengine - INFO - Epoch(train)  [99][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:43  time: 0.2259  data_time: 0.0077  memory: 3937  loss: 3.9268  loss_cls: 0.5584  loss_bbox: 1.9444  loss_obj: 0.7961  loss_l1: 0.6279
03/20 00:31:38 - mmengine - INFO - Epoch(train)  [99][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:32  time: 0.1873  data_time: 0.0077  memory: 3639  loss: 3.9223  loss_cls: 0.5615  loss_bbox: 1.9683  loss_obj: 0.8058  loss_l1: 0.5867
03/20 00:31:49 - mmengine - INFO - Epoch(train)  [99][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:22  time: 0.2086  data_time: 0.0078  memory: 3937  loss: 3.8094  loss_cls: 0.5497  loss_bbox: 1.9162  loss_obj: 0.7450  loss_l1: 0.5985
03/20 00:31:59 - mmengine - INFO - Epoch(train)  [99][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:11  time: 0.1984  data_time: 0.0078  memory: 3937  loss: 3.8094  loss_cls: 0.5475  loss_bbox: 1.9122  loss_obj: 0.7716  loss_l1: 0.5781
03/20 00:32:09 - mmengine - INFO - Epoch(train)  [99][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:05:00  time: 0.2167  data_time: 0.0077  memory: 3937  loss: 4.0363  loss_cls: 0.5672  loss_bbox: 2.0068  loss_obj: 0.8247  loss_l1: 0.6377
03/20 00:32:19 - mmengine - INFO - Epoch(train)  [99][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:04:49  time: 0.1940  data_time: 0.0078  memory: 3639  loss: 3.8444  loss_cls: 0.5511  loss_bbox: 1.9324  loss_obj: 0.7724  loss_l1: 0.5885
03/20 00:32:30 - mmengine - INFO - Epoch(train)  [99][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:04:39  time: 0.2127  data_time: 0.0077  memory: 3357  loss: 3.7954  loss_cls: 0.5417  loss_bbox: 1.9166  loss_obj: 0.7308  loss_l1: 0.6063
03/20 00:32:40 - mmengine - INFO - Epoch(train)  [99][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:04:28  time: 0.2015  data_time: 0.0077  memory: 3639  loss: 3.7820  loss_cls: 0.5444  loss_bbox: 1.9135  loss_obj: 0.7401  loss_l1: 0.5840
03/20 00:32:49 - mmengine - INFO - Epoch(train)  [99][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:04:17  time: 0.1733  data_time: 0.0077  memory: 2337  loss: 3.7712  loss_cls: 0.5542  loss_bbox: 1.9176  loss_obj: 0.7603  loss_l1: 0.5391
03/20 00:32:59 - mmengine - INFO - Epoch(train)  [99][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:04:06  time: 0.2174  data_time: 0.0078  memory: 3639  loss: 3.7200  loss_cls: 0.5329  loss_bbox: 1.8876  loss_obj: 0.7031  loss_l1: 0.5964
03/20 00:33:09 - mmengine - INFO - Epoch(train)  [99][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:56  time: 0.1960  data_time: 0.0076  memory: 3357  loss: 3.7770  loss_cls: 0.5436  loss_bbox: 1.9138  loss_obj: 0.7397  loss_l1: 0.5799
03/20 00:33:19 - mmengine - INFO - Epoch(train)  [99][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:45  time: 0.1993  data_time: 0.0077  memory: 3639  loss: 3.7493  loss_cls: 0.5450  loss_bbox: 1.8784  loss_obj: 0.7585  loss_l1: 0.5674
03/20 00:33:29 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:33:29 - mmengine - INFO - Epoch(train)  [99][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:34  time: 0.2031  data_time: 0.0076  memory: 3639  loss: 3.7976  loss_cls: 0.5428  loss_bbox: 1.9132  loss_obj: 0.7486  loss_l1: 0.5930
03/20 00:33:29 - mmengine - INFO - Saving checkpoint at 99 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:33:35 - mmengine - INFO - Epoch(val)  [99][ 50/250]    eta: 0:00:11  time: 0.0595  data_time: 0.0071  memory: 527  
03/20 00:33:38 - mmengine - INFO - Epoch(val)  [99][100/250]    eta: 0:00:08  time: 0.0587  data_time: 0.0065  memory: 527  
03/20 00:33:41 - mmengine - INFO - Epoch(val)  [99][150/250]    eta: 0:00:05  time: 0.0579  data_time: 0.0065  memory: 527  
03/20 00:33:43 - mmengine - INFO - Epoch(val)  [99][200/250]    eta: 0:00:02  time: 0.0581  data_time: 0.0064  memory: 527  
03/20 00:33:46 - mmengine - INFO - Epoch(val)  [99][250/250]    eta: 0:00:00  time: 0.0566  data_time: 0.0064  memory: 527  
03/20 00:33:47 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.19s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.67s).
Accumulating evaluation results...
DONE (t=0.81s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.327
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.680
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.256
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.257
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.389
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.553
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.361
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.460
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.634
03/20 00:33:52 - mmengine - INFO - bbox_mAP_copypaste: 0.327 0.680 0.256 0.257 0.389 0.553
03/20 00:33:52 - mmengine - INFO - Epoch(val) [99][250/250]    coco/bbox_mAP: 0.3270  coco/bbox_mAP_50: 0.6800  coco/bbox_mAP_75: 0.2560  coco/bbox_mAP_s: 0.2570  coco/bbox_mAP_m: 0.3890  coco/bbox_mAP_l: 0.5530  data_time: 0.0066  time: 0.0581
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/task_modules/assigners/sim_ota_assigner.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:34:01 - mmengine - INFO - Epoch(train) [100][  50/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:23  time: 0.1910  data_time: 0.0099  memory: 2817  loss: 3.7102  loss_cls: 0.5393  loss_bbox: 1.9003  loss_obj: 0.7269  loss_l1: 0.5436
03/20 00:34:11 - mmengine - INFO - Epoch(train) [100][ 100/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:13  time: 0.2024  data_time: 0.0077  memory: 3937  loss: 3.8074  loss_cls: 0.5424  loss_bbox: 1.9121  loss_obj: 0.7673  loss_l1: 0.5856
03/20 00:34:20 - mmengine - INFO - Epoch(train) [100][ 150/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:03:02  time: 0.1824  data_time: 0.0077  memory: 3357  loss: 3.8216  loss_cls: 0.5523  loss_bbox: 1.9483  loss_obj: 0.7582  loss_l1: 0.5628
03/20 00:34:30 - mmengine - INFO - Epoch(train) [100][ 200/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:02:51  time: 0.1921  data_time: 0.0078  memory: 3071  loss: 3.7428  loss_cls: 0.5439  loss_bbox: 1.9132  loss_obj: 0.7117  loss_l1: 0.5740
03/20 00:34:40 - mmengine - INFO - Epoch(train) [100][ 250/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:02:40  time: 0.2049  data_time: 0.0077  memory: 3937  loss: 3.8356  loss_cls: 0.5512  loss_bbox: 1.9251  loss_obj: 0.7679  loss_l1: 0.5915
03/20 00:34:51 - mmengine - INFO - Epoch(train) [100][ 300/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:02:30  time: 0.2099  data_time: 0.0077  memory: 3639  loss: 3.8293  loss_cls: 0.5508  loss_bbox: 1.9178  loss_obj: 0.7631  loss_l1: 0.5975
03/20 00:35:01 - mmengine - INFO - Epoch(train) [100][ 350/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:02:19  time: 0.2012  data_time: 0.0078  memory: 3639  loss: 3.8055  loss_cls: 0.5443  loss_bbox: 1.9182  loss_obj: 0.7517  loss_l1: 0.5913
03/20 00:35:11 - mmengine - INFO - Epoch(train) [100][ 400/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:02:08  time: 0.2031  data_time: 0.0077  memory: 3071  loss: 3.7346  loss_cls: 0.5464  loss_bbox: 1.8928  loss_obj: 0.7287  loss_l1: 0.5666
03/20 00:35:20 - mmengine - INFO - Epoch(train) [100][ 450/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:58  time: 0.1751  data_time: 0.0077  memory: 2817  loss: 3.8288  loss_cls: 0.5525  loss_bbox: 1.9583  loss_obj: 0.7615  loss_l1: 0.5565
03/20 00:35:29 - mmengine - INFO - Epoch(train) [100][ 500/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:47  time: 0.1871  data_time: 0.0078  memory: 3639  loss: 3.7882  loss_cls: 0.5472  loss_bbox: 1.9089  loss_obj: 0.7662  loss_l1: 0.5659
03/20 00:35:40 - mmengine - INFO - Epoch(train) [100][ 550/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:36  time: 0.2057  data_time: 0.0076  memory: 3937  loss: 3.7502  loss_cls: 0.5369  loss_bbox: 1.8854  loss_obj: 0.7431  loss_l1: 0.5848
03/20 00:35:49 - mmengine - INFO - Epoch(train) [100][ 600/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:25  time: 0.1871  data_time: 0.0077  memory: 3639  loss: 3.9304  loss_cls: 0.5589  loss_bbox: 1.9648  loss_obj: 0.8265  loss_l1: 0.5803
03/20 00:35:59 - mmengine - INFO - Epoch(train) [100][ 650/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:15  time: 0.2001  data_time: 0.0077  memory: 3357  loss: 3.8408  loss_cls: 0.5483  loss_bbox: 1.9267  loss_obj: 0.7698  loss_l1: 0.5960
03/20 00:36:08 - mmengine - INFO - Epoch(train) [100][ 700/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:01:04  time: 0.1894  data_time: 0.0077  memory: 2817  loss: 3.7892  loss_cls: 0.5519  loss_bbox: 1.9244  loss_obj: 0.7465  loss_l1: 0.5664
03/20 00:36:17 - mmengine - INFO - Epoch(train) [100][ 750/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:53  time: 0.1785  data_time: 0.0077  memory: 3071  loss: 3.7751  loss_cls: 0.5611  loss_bbox: 1.9277  loss_obj: 0.7298  loss_l1: 0.5565
03/20 00:36:28 - mmengine - INFO - Epoch(train) [100][ 800/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:42  time: 0.2068  data_time: 0.0078  memory: 3937  loss: 3.8447  loss_cls: 0.5548  loss_bbox: 1.9303  loss_obj: 0.7601  loss_l1: 0.5995
03/20 00:36:38 - mmengine - INFO - Epoch(train) [100][ 850/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:32  time: 0.2152  data_time: 0.0078  memory: 3639  loss: 3.8105  loss_cls: 0.5492  loss_bbox: 1.9181  loss_obj: 0.7355  loss_l1: 0.6077
03/20 00:36:50 - mmengine - INFO - Epoch(train) [100][ 900/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:21  time: 0.2365  data_time: 0.0078  memory: 3937  loss: 3.8653  loss_cls: 0.5463  loss_bbox: 1.9363  loss_obj: 0.7461  loss_l1: 0.6365
03/20 00:37:01 - mmengine - INFO - Epoch(train) [100][ 950/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:10  time: 0.2231  data_time: 0.0077  memory: 3937  loss: 3.8489  loss_cls: 0.5423  loss_bbox: 1.9111  loss_obj: 0.7790  loss_l1: 0.6165
03/20 00:37:12 - mmengine - INFO - Exp name: yolox_s_lite_20250319_175030
03/20 00:37:12 - mmengine - INFO - Epoch(train) [100][1000/1000]  base_lr: 1.6254e-03 lr: 1.6254e-03  eta: 0:00:00  time: 0.2138  data_time: 0.0077  memory: 3639  loss: 3.7951  loss_cls: 0.5471  loss_bbox: 1.9118  loss_obj: 0.7208  loss_l1: 0.6153
03/20 00:37:12 - mmengine - INFO - Saving checkpoint at 100 epochs
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
03/20 00:37:17 - mmengine - INFO - Epoch(val) [100][ 50/250]    eta: 0:00:11  time: 0.0592  data_time: 0.0072  memory: 527  
03/20 00:37:20 - mmengine - INFO - Epoch(val) [100][100/250]    eta: 0:00:08  time: 0.0583  data_time: 0.0065  memory: 527  
03/20 00:37:23 - mmengine - INFO - Epoch(val) [100][150/250]    eta: 0:00:05  time: 0.0584  data_time: 0.0065  memory: 527  
03/20 00:37:26 - mmengine - INFO - Epoch(val) [100][200/250]    eta: 0:00:02  time: 0.0578  data_time: 0.0065  memory: 527  
03/20 00:37:29 - mmengine - INFO - Epoch(val) [100][250/250]    eta: 0:00:00  time: 0.0570  data_time: 0.0065  memory: 527  
03/20 00:37:30 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.18s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.61s).
Accumulating evaluation results...
DONE (t=0.80s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.331
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.686
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.262
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.261
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.393
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.552
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.422
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.422
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.422
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.366
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.464
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.634
03/20 00:37:34 - mmengine - INFO - bbox_mAP_copypaste: 0.331 0.686 0.262 0.261 0.393 0.552
03/20 00:37:34 - mmengine - INFO - Epoch(val) [100][250/250]    coco/bbox_mAP: 0.3310  coco/bbox_mAP_50: 0.6860  coco/bbox_mAP_75: 0.2620  coco/bbox_mAP_s: 0.2610  coco/bbox_mAP_m: 0.3930  coco/bbox_mAP_l: 0.5520  data_time: 0.0066  time: 0.0581
03/20 00:37:35 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/20 00:37:35 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/20 00:37:35 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
03/20 00:37:35 - mmengine - INFO - Export PyTorch model to ONNX: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx.
/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/mmdet/models/backbones/csp_darknet.py:244: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
/local_data/home/mattmak/.pyenv/versions/benchmark_cuda/lib/python3.10/site-packages/torch/onnx/symbolic_opset9.py:5383: UserWarning: Exporting aten::index operator of advanced indexing in opset 17 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  warnings.warn(
03/20 00:37:40 - mmengine - INFO - Execute onnx optimize passes.
Converted model is valid!
[rank0]:[W320 00:37:42.560381682 ProcessGroupNCCL.cpp:1496] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
['_base_   = ["/local_data/home/mattmak/edgeai-tensorlab/edgeai-mmdetection/configs_edgeailite/yolox/yolox_s_lite.py"]\n', 'work_dir   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training"', "classes = ('objects', 'biker', 'car', 'pedestrian', 'trafficLight', 'trafficLight-Green', 'trafficLight-GreenLeft', 'trafficLight-Red', 'trafficLight-RedLeft', 'trafficLight-Yellow', 'trafficLight-YellowLeft', 'truck')", 'data_root   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset"', 'max_epochs   = 100', 'export_onnx_model=True', 'optim_wrapper=dict(\n  optimizer=dict(\n    lr=0.005\n  ) \n) \n', 'model=dict(\n  bbox_head=dict(\n    num_classes=12\n  ) \n) \n', 'train_dataloader=dict(\n  dataset=dict(\n      dataset=dict(\n      data_root   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset",\n          ann_file = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_train.json",\n          data_prefix = dict(img="train/"),\n          metainfo=dict(classes=classes),\n          ) \n      ) \n  ) \n', 'train_dataset=dict(\n      dataset=dict(\n          data_root   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset",\n          ann_file = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_train.json",\n          data_prefix = dict(img="train/"),\n          metainfo=dict(classes=classes),\n      ) \n  ) \n', 'test_dataloader=dict(\n  dataset=dict(\n      data_root   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset",\n      ann_file = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json",\n      data_prefix = dict(img="val/"),\n          metainfo=dict(classes=classes),\n    ) \n) \n', 'val_dataloader=dict(\n  dataset=dict(\n      data_root   = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset",\n      ann_file = "/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json",\n      data_prefix = dict(img="val/"),\n      metainfo=dict(classes=classes),\n    ) \n) \n', 'test_evaluator = dict( \n    ann_file="/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json") \n', 'val_evaluator = dict( \n    ann_file="/local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/dataset/annotations/instances_val.json") \n', 'train_cfg = dict(max_epochs=100, type="EpochBasedTrainLoop", val_interval=1)\n', 'load_from   = "./data/downloads/pretrained/yolox_s_lite/yolox_s_lite_640x640_20220221_checkpoint.pth"', 'find_unused_parameters=True']
Trained model is at: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training

SUCCESS: ModelMaker - Training completed.

INFO:20250320-003745: model import is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20250320-003745: parallel_run - parallel_processes:1 parallel_devices=[0]


TASKS                                                       |   0%|          || 0/1 [00:00<?, ?it/s]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20250320-003745: starting process on parallel_device - 0

INFO:20250320-003745: starting - od-8220
INFO:20250320-003745: model_path - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx
INFO:20250320-003745: model_file - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20250320-003745: quant_file - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt
Downloading 1/1: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx
Download done for /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx
Downloading 1/1: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx
Download done for /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx

INFO:20250320-003745: running - od-8220
INFO:20250320-003745: pipeline_config - {'task_type': 'detection', 'dataset_category': 'coco', 'calibration_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f8e931831c0>, 'input_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f8e931812d0>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0x7f8e8dc04a30>, 'session': <edgeai_benchmark.sessions.onnxrt_session.ONNXRTSession object at 0x7f8e8dc04a90>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0x7f8e8dc04d30>, 'metric': {'label_offset_pred': 0}, 'model_info': {'metric_reference': {'accuracy_ap[.5:.95]%': None}, 'model_shortlist': 10, 'compact_name': 'yolox-s-lite-mmdet-coco-640x640', 'shortlisted': True, 'recommended': True}}
INFO:20250320-003745: import  - od-8220 - this may take some time...
INFO:20250320-004216: import completed  - od-8220 - 271 sec

SUCCESS:20250320-004216: benchmark results - {}

INFO:20250320-003845: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-003945: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004045: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004145: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0


TASKS                                                       | 100%|██████████|| 1/1 [04:31<00:00, 271.09s/it]
TASKS                                                       | 100%|██████████|| 1/1 [04:31<00:00, 271.19s/it]



INFO:20250320-004216: model inference is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20250320-004216: parallel_run - parallel_processes:1 parallel_devices=[0]


TASKS                                                       |   0%|          || 0/1 [00:00<?, ?it/s]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20250320-004216: starting process on parallel_device - 0

INFO:20250320-004216: starting - od-8220
INFO:20250320-004216: model_path - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/training/model.onnx
INFO:20250320-004216: model_file - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20250320-004216: quant_file - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt

INFO:20250320-004216: running - od-8220
INFO:20250320-004216: pipeline_config - {'task_type': 'detection', 'dataset_category': 'coco', 'calibration_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f8e931831c0>, 'input_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f8e931812d0>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0x7f8e8dc04a30>, 'session': <edgeai_benchmark.sessions.onnxrt_session.ONNXRTSession object at 0x7f8e8dc04a90>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0x7f8e8dc04d30>, 'metric': {'label_offset_pred': 0}, 'model_info': {'metric_reference': {'accuracy_ap[.5:.95]%': None}, 'model_shortlist': 10, 'compact_name': 'yolox-s-lite-mmdet-coco-640x640', 'shortlisted': True, 'recommended': True}}
INFO:20250320-004216: infer  - od-8220 - this may take some time...
infer : od-8220                                             |   0%|          || 0/2000 [00:00<?, ?it/s]
infer : od-8220                                             |          |     0% 0/2000| [< ]libtidl_onnxrt_EP loaded 0x562494b0e610 
Final number of subgraphs created are : 1, - Offloaded Nodes - 271, Total Nodes - 271 
The soft limit is 10240
The hard limit is 10240
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.8s:  VX_ZONE_ERROR:Enabled
 0.9s:  VX_ZONE_WARNING:Enabled
 0.1170s:  VX_ZONE_INIT:[tivxInit:190] Initialization Done !!!

infer : od-8220                                             |   0%|          || 1/2000 [00:12<6:53:21, 12.41s/it]
infer : od-8220                                             |   0%|          || 2/2000 [00:24<6:51:44, 12.36s/it]
infer : od-8220                                             |   0%|          || 3/2000 [00:37<6:51:05, 12.35s/it]
infer : od-8220                                             |   0%|          || 4/2000 [00:49<6:50:45, 12.35s/it]
infer : od-8220                                             |   0%|          || 5/2000 [01:01<6:50:44, 12.35s/it]
infer : od-8220                                             |   0%|          || 6/2000 [01:14<6:50:18, 12.35s/it]
infer : od-8220                                             |   0%|          || 7/2000 [01:26<6:49:57, 12.34s/it]
infer : od-8220                                             |   0%|          || 8/2000 [01:38<6:49:43, 12.34s/it]
infer : od-8220                                             |   0%|          || 9/2000 [01:51<6:49:18, 12.33s/it]
infer : od-8220                                             |   0%|          || 10/2000 [02:03<6:49:07, 12.34s/it]
infer : od-8220                                             |   1%|          || 11/2000 [02:15<6:49:01, 12.34s/it]
infer : od-8220                                             |   1%|          || 12/2000 [02:28<6:48:51, 12.34s/it]
infer : od-8220                                             |   1%|          || 13/2000 [02:40<6:48:27, 12.33s/it]
infer : od-8220                                             |   1%|          || 14/2000 [02:52<6:48:06, 12.33s/it]
infer : od-8220                                             |   1%|          || 15/2000 [03:05<6:47:57, 12.33s/it]
infer : od-8220                                             |   1%|          || 16/2000 [03:17<6:47:50, 12.33s/it]
infer : od-8220                                             |   1%|          || 17/2000 [03:29<6:47:52, 12.34s/it]
infer : od-8220                                             |   1%|          || 18/2000 [03:42<6:47:42, 12.34s/it]
infer : od-8220                                             |   1%|          || 19/2000 [03:54<6:47:24, 12.34s/it]
infer : od-8220                                             |   1%|1         || 20/2000 [04:06<6:47:10, 12.34s/it]
infer : od-8220                                             |   1%|1         || 21/2000 [04:19<6:46:47, 12.33s/it]
infer : od-8220                                             |   1%|1         || 22/2000 [04:31<6:46:49, 12.34s/it]
infer : od-8220                                             |   1%|1         || 23/2000 [04:43<6:46:42, 12.34s/it]
infer : od-8220                                             |   1%|1         || 24/2000 [04:56<6:46:29, 12.34s/it]
infer : od-8220                                             |   1%|1         || 25/2000 [05:08<6:46:02, 12.34s/it]
infer : od-8220                                             |   1%|1         || 26/2000 [05:20<6:45:43, 12.33s/it]
infer : od-8220                                             |   1%|1         || 27/2000 [05:33<6:45:33, 12.33s/it]
infer : od-8220                                             |   1%|1         || 28/2000 [05:45<6:45:32, 12.34s/it]
infer : od-8220                                             |   1%|1         || 29/2000 [05:57<6:45:27, 12.34s/it]
infer : od-8220                                             |   2%|1         || 30/2000 [06:10<6:45:24, 12.35s/it]
infer : od-8220                                             |   2%|1         || 31/2000 [06:22<6:44:59, 12.34s/it]
infer : od-8220                                             |   2%|1         || 32/2000 [06:34<6:44:38, 12.34s/it]
infer : od-8220                                             |   2%|1         || 33/2000 [06:47<6:44:24, 12.34s/it]
infer : od-8220                                             |   2%|1         || 34/2000 [06:59<6:44:12, 12.34s/it]
infer : od-8220                                             |   2%|1         || 35/2000 [07:11<6:43:57, 12.33s/it]
infer : od-8220                                             |   2%|1         || 36/2000 [07:24<6:43:58, 12.34s/it]
infer : od-8220                                             |   2%|1         || 37/2000 [07:36<6:43:43, 12.34s/it]
infer : od-8220                                             |   2%|1         || 38/2000 [07:48<6:43:12, 12.33s/it]
infer : od-8220                                             |   2%|1         || 39/2000 [08:01<6:43:09, 12.34s/it]
infer : od-8220                                             |   2%|2         || 40/2000 [08:13<6:42:59, 12.34s/it]
infer : od-8220                                             |   2%|2         || 41/2000 [08:25<6:42:39, 12.33s/it]
infer : od-8220                                             |   2%|2         || 42/2000 [08:38<6:42:29, 12.33s/it]
infer : od-8220                                             |   2%|2         || 43/2000 [08:50<6:42:03, 12.33s/it]
infer : od-8220                                             |   2%|2         || 44/2000 [09:02<6:41:59, 12.33s/it]
infer : od-8220                                             |   2%|2         || 45/2000 [09:15<6:41:47, 12.33s/it]
infer : od-8220                                             |   2%|2         || 46/2000 [09:27<6:41:47, 12.34s/it]
infer : od-8220                                             |   2%|2         || 47/2000 [09:39<6:41:41, 12.34s/it]
infer : od-8220                                             |   2%|2         || 48/2000 [09:52<6:41:28, 12.34s/it]
infer : od-8220                                             |   2%|2         || 49/2000 [10:04<6:41:16, 12.34s/it]
infer : od-8220                                             |   2%|2         || 50/2000 [10:16<6:40:46, 12.33s/it]
infer : od-8220                                             |   3%|2         || 51/2000 [10:29<6:40:38, 12.33s/it]
infer : od-8220                                             |   3%|2         || 52/2000 [10:41<6:40:34, 12.34s/it]
infer : od-8220                                             |   3%|2         || 53/2000 [10:53<6:40:11, 12.33s/it]
infer : od-8220                                             |   3%|2         || 54/2000 [11:06<6:39:57, 12.33s/it]
infer : od-8220                                             |   3%|2         || 55/2000 [11:18<6:39:35, 12.33s/it]
infer : od-8220                                             |   3%|2         || 56/2000 [11:30<6:39:13, 12.32s/it]
infer : od-8220                                             |   3%|2         || 57/2000 [11:43<6:39:17, 12.33s/it]
infer : od-8220                                             |   3%|2         || 58/2000 [11:55<6:39:13, 12.33s/it]
infer : od-8220                                             |   3%|2         || 59/2000 [12:07<6:38:56, 12.33s/it]
infer : od-8220                                             |   3%|3         || 60/2000 [12:20<6:38:30, 12.33s/it]
infer : od-8220                                             |   3%|3         || 61/2000 [12:32<6:38:05, 12.32s/it]
infer : od-8220                                             |   3%|3         || 62/2000 [12:44<6:38:02, 12.32s/it]
infer : od-8220                                             |   3%|3         || 63/2000 [12:57<6:37:58, 12.33s/it]
infer : od-8220                                             |   3%|3         || 64/2000 [13:09<6:37:44, 12.33s/it]
infer : od-8220                                             |   3%|3         || 65/2000 [13:21<6:37:31, 12.33s/it]
infer : od-8220                                             |   3%|3         || 66/2000 [13:34<6:37:16, 12.33s/it]
infer : od-8220                                             |   3%|3         || 67/2000 [13:46<6:37:00, 12.32s/it]
infer : od-8220                                             |   3%|3         || 68/2000 [13:58<6:37:04, 12.33s/it]
infer : od-8220                                             |   3%|3         || 69/2000 [14:11<6:36:54, 12.33s/it]
infer : od-8220                                             |   4%|3         || 70/2000 [14:23<6:36:40, 12.33s/it]
infer : od-8220                                             |   4%|3         || 71/2000 [14:35<6:36:25, 12.33s/it]
infer : od-8220                                             |   4%|3         || 72/2000 [14:48<6:36:13, 12.33s/it]
infer : od-8220                                             |   4%|3         || 73/2000 [15:00<6:35:54, 12.33s/it]
infer : od-8220                                             |   4%|3         || 74/2000 [15:12<6:35:52, 12.33s/it]
infer : od-8220                                             |   4%|3         || 75/2000 [15:25<6:35:41, 12.33s/it]
infer : od-8220                                             |   4%|3         || 76/2000 [15:37<6:35:49, 12.34s/it]
infer : od-8220                                             |   4%|3         || 77/2000 [15:49<6:35:19, 12.33s/it]
infer : od-8220                                             |   4%|3         || 78/2000 [16:02<6:35:10, 12.34s/it]
infer : od-8220                                             |   4%|3         || 79/2000 [16:14<6:34:42, 12.33s/it]
infer : od-8220                                             |   4%|4         || 80/2000 [16:26<6:34:39, 12.33s/it]
infer : od-8220                                             |   4%|4         || 81/2000 [16:39<6:34:27, 12.33s/it]
infer : od-8220                                             |   4%|4         || 82/2000 [16:51<6:34:08, 12.33s/it]
infer : od-8220                                             |   4%|4         || 83/2000 [17:03<6:33:57, 12.33s/it]
infer : od-8220                                             |   4%|4         || 84/2000 [17:16<6:34:00, 12.34s/it]
infer : od-8220                                             |   4%|4         || 85/2000 [17:28<6:33:56, 12.34s/it]
infer : od-8220                                             |   4%|4         || 86/2000 [17:40<6:33:50, 12.35s/it]
infer : od-8220                                             |   4%|4         || 87/2000 [17:53<6:33:41, 12.35s/it]
infer : od-8220                                             |   4%|4         || 88/2000 [18:05<6:33:26, 12.35s/it]
infer : od-8220                                             |   4%|4         || 89/2000 [18:17<6:33:02, 12.34s/it]
infer : od-8220                                             |   4%|4         || 90/2000 [18:30<6:32:48, 12.34s/it]
infer : od-8220                                             |   5%|4         || 91/2000 [18:42<6:32:27, 12.33s/it]
infer : od-8220                                             |   5%|4         || 92/2000 [18:54<6:32:25, 12.34s/it]
infer : od-8220                                             |   5%|4         || 93/2000 [19:07<6:32:12, 12.34s/it]
infer : od-8220                                             |   5%|4         || 94/2000 [19:19<6:31:53, 12.34s/it]
infer : od-8220                                             |   5%|4         || 95/2000 [19:31<6:31:26, 12.33s/it]
infer : od-8220                                             |   5%|4         || 96/2000 [19:44<6:31:05, 12.32s/it]
infer : od-8220                                             |   5%|4         || 97/2000 [19:56<6:30:50, 12.32s/it]
infer : od-8220                                             |   5%|4         || 98/2000 [20:08<6:30:46, 12.33s/it]
infer : od-8220                                             |   5%|4         || 99/2000 [20:21<6:30:42, 12.33s/it]
infer : od-8220                                             |   5%|5         || 100/2000 [20:33<6:30:38, 12.34s/it]
infer : od-8220                                             |   5%|5         || 101/2000 [20:45<6:30:12, 12.33s/it]
infer : od-8220                                             |   5%|5         || 102/2000 [20:58<6:30:01, 12.33s/it]
infer : od-8220                                             |   5%|5         || 103/2000 [21:10<6:29:43, 12.33s/it]
infer : od-8220                                             |   5%|5         || 104/2000 [21:22<6:29:35, 12.33s/it]
infer : od-8220                                             |   5%|5         || 105/2000 [21:35<6:29:24, 12.33s/it]
infer : od-8220                                             |   5%|5         || 106/2000 [21:47<6:28:58, 12.32s/it]
infer : od-8220                                             |   5%|5         || 107/2000 [21:59<6:28:46, 12.32s/it]
infer : od-8220                                             |   5%|5         || 108/2000 [22:12<6:28:31, 12.32s/it]
infer : od-8220                                             |   5%|5         || 109/2000 [22:24<6:28:18, 12.32s/it]
infer : od-8220                                             |   6%|5         || 110/2000 [22:36<6:28:14, 12.33s/it]
infer : od-8220                                             |   6%|5         || 111/2000 [22:49<6:28:05, 12.33s/it]
infer : od-8220                                             |   6%|5         || 112/2000 [23:01<6:27:50, 12.33s/it]
infer : od-8220                                             |   6%|5         || 113/2000 [23:13<6:27:42, 12.33s/it]
infer : od-8220                                             |   6%|5         || 114/2000 [23:26<6:27:25, 12.33s/it]
infer : od-8220                                             |   6%|5         || 115/2000 [23:38<6:27:17, 12.33s/it]
infer : od-8220                                             |   6%|5         || 116/2000 [23:50<6:27:18, 12.33s/it]
infer : od-8220                                             |   6%|5         || 117/2000 [24:03<6:27:03, 12.33s/it]
infer : od-8220                                             |   6%|5         || 118/2000 [24:15<6:26:46, 12.33s/it]
infer : od-8220                                             |   6%|5         || 119/2000 [24:27<6:26:38, 12.33s/it]
infer : od-8220                                             |   6%|6         || 120/2000 [24:40<6:26:37, 12.34s/it]
infer : od-8220                                             |   6%|6         || 121/2000 [24:52<6:26:22, 12.34s/it]
infer : od-8220                                             |   6%|6         || 122/2000 [25:04<6:26:21, 12.34s/it]
infer : od-8220                                             |   6%|6         || 123/2000 [25:17<6:26:09, 12.34s/it]
infer : od-8220                                             |   6%|6         || 124/2000 [25:29<6:25:51, 12.34s/it]
infer : od-8220                                             |   6%|6         || 125/2000 [25:41<6:25:48, 12.35s/it]
infer : od-8220                                             |   6%|6         || 126/2000 [25:54<6:25:39, 12.35s/it]
infer : od-8220                                             |   6%|6         || 127/2000 [26:06<6:25:20, 12.34s/it]
infer : od-8220                                             |   6%|6         || 128/2000 [26:18<6:25:09, 12.34s/it]
infer : od-8220                                             |   6%|6         || 129/2000 [26:31<6:24:55, 12.34s/it]
infer : od-8220                                             |   6%|6         || 130/2000 [26:43<6:24:38, 12.34s/it]
infer : od-8220                                             |   7%|6         || 131/2000 [26:55<6:24:12, 12.33s/it]
infer : od-8220                                             |   7%|6         || 132/2000 [27:08<6:23:57, 12.33s/it]
infer : od-8220                                             |   7%|6         || 133/2000 [27:20<6:23:40, 12.33s/it]
infer : od-8220                                             |   7%|6         || 134/2000 [27:32<6:23:31, 12.33s/it]
infer : od-8220                                             |   7%|6         || 135/2000 [27:45<6:23:17, 12.33s/it]
infer : od-8220                                             |   7%|6         || 136/2000 [27:57<6:22:49, 12.32s/it]
infer : od-8220                                             |   7%|6         || 137/2000 [28:09<6:22:36, 12.32s/it]
infer : od-8220                                             |   7%|6         || 138/2000 [28:22<6:22:16, 12.32s/it]
infer : od-8220                                             |   7%|6         || 139/2000 [28:34<6:22:12, 12.32s/it]
infer : od-8220                                             |   7%|7         || 140/2000 [28:46<6:22:22, 12.33s/it]
infer : od-8220                                             |   7%|7         || 141/2000 [28:59<6:22:26, 12.34s/it]
infer : od-8220                                             |   7%|7         || 142/2000 [29:11<6:22:06, 12.34s/it]
infer : od-8220                                             |   7%|7         || 143/2000 [29:23<6:21:34, 12.33s/it]
infer : od-8220                                             |   7%|7         || 144/2000 [29:36<6:21:27, 12.33s/it]
infer : od-8220                                             |   7%|7         || 145/2000 [29:48<6:21:11, 12.33s/it]
infer : od-8220                                             |   7%|7         || 146/2000 [30:00<6:20:48, 12.32s/it]
infer : od-8220                                             |   7%|7         || 147/2000 [30:13<6:20:27, 12.32s/it]
infer : od-8220                                             |   7%|7         || 148/2000 [30:25<6:19:59, 12.31s/it]
infer : od-8220                                             |   7%|7         || 149/2000 [30:37<6:19:58, 12.32s/it]
infer : od-8220                                             |   8%|7         || 150/2000 [30:50<6:20:02, 12.33s/it]
infer : od-8220                                             |   8%|7         || 151/2000 [31:02<6:20:05, 12.33s/it]
infer : od-8220                                             |   8%|7         || 152/2000 [31:14<6:19:48, 12.33s/it]
infer : od-8220                                             |   8%|7         || 153/2000 [31:27<6:19:21, 12.32s/it]
infer : od-8220                                             |   8%|7         || 154/2000 [31:39<6:19:13, 12.33s/it]
infer : od-8220                                             |   8%|7         || 155/2000 [31:51<6:19:29, 12.34s/it]
infer : od-8220                                             |   8%|7         || 156/2000 [32:04<6:19:11, 12.34s/it]
infer : od-8220                                             |   8%|7         || 157/2000 [32:16<6:18:54, 12.34s/it]
infer : od-8220                                             |   8%|7         || 158/2000 [32:28<6:18:39, 12.33s/it]
infer : od-8220                                             |   8%|7         || 159/2000 [32:41<6:18:25, 12.33s/it]
infer : od-8220                                             |   8%|8         || 160/2000 [32:53<6:18:07, 12.33s/it]
infer : od-8220                                             |   8%|8         || 161/2000 [33:05<6:17:53, 12.33s/it]
infer : od-8220                                             |   8%|8         || 162/2000 [33:18<6:17:32, 12.32s/it]
infer : od-8220                                             |   8%|8         || 163/2000 [33:30<6:17:14, 12.32s/it]
infer : od-8220                                             |   8%|8         || 164/2000 [33:42<6:17:13, 12.33s/it]
infer : od-8220                                             |   8%|8         || 165/2000 [33:54<6:16:40, 12.32s/it]
infer : od-8220                                             |   8%|8         || 166/2000 [34:07<6:16:17, 12.31s/it]
infer : od-8220                                             |   8%|8         || 167/2000 [34:19<6:16:06, 12.31s/it]
infer : od-8220                                             |   8%|8         || 168/2000 [34:31<6:15:57, 12.31s/it]
infer : od-8220                                             |   8%|8         || 169/2000 [34:44<6:16:18, 12.33s/it]
infer : od-8220                                             |   8%|8         || 170/2000 [34:56<6:16:06, 12.33s/it]
infer : od-8220                                             |   9%|8         || 171/2000 [35:08<6:15:50, 12.33s/it]
infer : od-8220                                             |   9%|8         || 172/2000 [35:21<6:15:32, 12.33s/it]
infer : od-8220                                             |   9%|8         || 173/2000 [35:33<6:15:16, 12.32s/it]
infer : od-8220                                             |   9%|8         || 174/2000 [35:45<6:15:10, 12.33s/it]
infer : od-8220                                             |   9%|8         || 175/2000 [35:58<6:15:19, 12.34s/it]
infer : od-8220                                             |   9%|8         || 176/2000 [36:10<6:14:55, 12.33s/it]
infer : od-8220                                             |   9%|8         || 177/2000 [36:22<6:14:29, 12.33s/it]
infer : od-8220                                             |   9%|8         || 178/2000 [36:35<6:14:15, 12.32s/it]
infer : od-8220                                             |   9%|8         || 179/2000 [36:47<6:13:53, 12.32s/it]
infer : od-8220                                             |   9%|9         || 180/2000 [36:59<6:13:39, 12.32s/it]
infer : od-8220                                             |   9%|9         || 181/2000 [37:12<6:13:33, 12.32s/it]
infer : od-8220                                             |   9%|9         || 182/2000 [37:24<6:13:25, 12.32s/it]
infer : od-8220                                             |   9%|9         || 183/2000 [37:36<6:13:06, 12.32s/it]
infer : od-8220                                             |   9%|9         || 184/2000 [37:49<6:13:09, 12.33s/it]
infer : od-8220                                             |   9%|9         || 185/2000 [38:01<6:12:49, 12.32s/it]
infer : od-8220                                             |   9%|9         || 186/2000 [38:13<6:12:51, 12.33s/it]
infer : od-8220                                             |   9%|9         || 187/2000 [38:26<6:12:44, 12.34s/it]
infer : od-8220                                             |   9%|9         || 188/2000 [38:38<6:12:22, 12.33s/it]
infer : od-8220                                             |   9%|9         || 189/2000 [38:50<6:11:59, 12.32s/it]
infer : od-8220                                             |  10%|9         || 190/2000 [39:03<6:11:36, 12.32s/it]
infer : od-8220                                             |  10%|9         || 191/2000 [39:15<6:11:40, 12.33s/it]
infer : od-8220                                             |  10%|9         || 192/2000 [39:27<6:11:15, 12.32s/it]
infer : od-8220                                             |  10%|9         || 193/2000 [39:40<6:10:55, 12.32s/it]
infer : od-8220                                             |  10%|9         || 194/2000 [39:52<6:10:35, 12.31s/it]
infer : od-8220                                             |  10%|9         || 195/2000 [40:04<6:10:09, 12.30s/it]
infer : od-8220                                             |  10%|9         || 196/2000 [40:16<6:09:51, 12.30s/it]
infer : od-8220                                             |  10%|9         || 197/2000 [40:29<6:09:45, 12.30s/it]
infer : od-8220                                             |  10%|9         || 198/2000 [40:41<6:09:41, 12.31s/it]
infer : od-8220                                             |  10%|9         || 199/2000 [40:53<6:09:52, 12.32s/it]
infer : od-8220                                             |  10%|#         || 200/2000 [41:06<6:09:31, 12.32s/it]
infer : od-8220                                             |  10%|#         || 201/2000 [41:18<6:09:11, 12.31s/it]
infer : od-8220                                             |  10%|#         || 202/2000 [41:30<6:08:42, 12.30s/it]
infer : od-8220                                             |  10%|#         || 203/2000 [41:43<6:08:47, 12.31s/it]
infer : od-8220                                             |  10%|#         || 204/2000 [41:55<6:08:25, 12.31s/it]
infer : od-8220                                             |  10%|#         || 205/2000 [42:07<6:08:15, 12.31s/it]
infer : od-8220                                             |  10%|#         || 206/2000 [42:20<6:07:50, 12.30s/it]
infer : od-8220                                             |  10%|#         || 207/2000 [42:32<6:07:35, 12.30s/it]
infer : od-8220                                             |  10%|#         || 208/2000 [42:44<6:07:26, 12.30s/it]
infer : od-8220                                             |  10%|#         || 209/2000 [42:56<6:07:05, 12.30s/it]
infer : od-8220                                             |  10%|#         || 210/2000 [43:09<6:06:46, 12.29s/it]
infer : od-8220                                             |  11%|#         || 211/2000 [43:21<6:06:14, 12.28s/it]
infer : od-8220                                             |  11%|#         || 212/2000 [43:33<6:06:16, 12.29s/it]
infer : od-8220                                             |  11%|#         || 213/2000 [43:46<6:06:25, 12.30s/it]
infer : od-8220                                             |  11%|#         || 214/2000 [43:58<6:06:04, 12.30s/it]
infer : od-8220                                             |  11%|#         || 215/2000 [44:10<6:06:06, 12.31s/it]
infer : od-8220                                             |  11%|#         || 216/2000 [44:23<6:06:02, 12.31s/it]
infer : od-8220                                             |  11%|#         || 217/2000 [44:35<6:06:07, 12.32s/it]
infer : od-8220                                             |  11%|#         || 218/2000 [44:47<6:05:56, 12.32s/it]
infer : od-8220                                             |  11%|#         || 219/2000 [45:00<6:05:32, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 220/2000 [45:12<6:05:13, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 221/2000 [45:24<6:04:54, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 222/2000 [45:36<6:04:51, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 223/2000 [45:49<6:04:32, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 224/2000 [46:01<6:04:26, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 225/2000 [46:13<6:04:17, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 226/2000 [46:26<6:04:02, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 227/2000 [46:38<6:03:40, 12.31s/it]
infer : od-8220                                             |  11%|#1        || 228/2000 [46:50<6:03:44, 12.32s/it]
infer : od-8220                                             |  11%|#1        || 229/2000 [47:03<6:03:29, 12.31s/it]
infer : od-8220                                             |  12%|#1        || 230/2000 [47:15<6:03:14, 12.31s/it]
infer : od-8220                                             |  12%|#1        || 231/2000 [47:27<6:03:25, 12.33s/it]
infer : od-8220                                             |  12%|#1        || 232/2000 [47:40<6:03:33, 12.34s/it]
infer : od-8220                                             |  12%|#1        || 233/2000 [47:52<6:03:27, 12.34s/it]
infer : od-8220                                             |  12%|#1        || 234/2000 [48:04<6:03:13, 12.34s/it]
infer : od-8220                                             |  12%|#1        || 235/2000 [48:17<6:02:44, 12.33s/it]
infer : od-8220                                             |  12%|#1        || 236/2000 [48:29<6:02:08, 12.32s/it]
infer : od-8220                                             |  12%|#1        || 237/2000 [48:41<6:02:08, 12.32s/it]
infer : od-8220                                             |  12%|#1        || 238/2000 [48:54<6:01:58, 12.33s/it]
infer : od-8220                                             |  12%|#1        || 239/2000 [49:06<6:01:41, 12.32s/it]
infer : od-8220                                             |  12%|#2        || 240/2000 [49:18<6:01:27, 12.32s/it]
infer : od-8220                                             |  12%|#2        || 241/2000 [49:31<6:01:17, 12.32s/it]
infer : od-8220                                             |  12%|#2        || 242/2000 [49:43<6:01:24, 12.33s/it]
infer : od-8220                                             |  12%|#2        || 243/2000 [49:55<6:01:24, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 244/2000 [50:08<6:01:07, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 245/2000 [50:20<6:01:01, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 246/2000 [50:32<6:00:47, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 247/2000 [50:45<6:00:20, 12.33s/it]
infer : od-8220                                             |  12%|#2        || 248/2000 [50:57<6:00:14, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 249/2000 [51:09<6:00:08, 12.34s/it]
infer : od-8220                                             |  12%|#2        || 250/2000 [51:22<5:59:58, 12.34s/it]
infer : od-8220                                             |  13%|#2        || 251/2000 [51:34<5:59:38, 12.34s/it]
infer : od-8220                                             |  13%|#2        || 252/2000 [51:46<5:59:21, 12.33s/it]
infer : od-8220                                             |  13%|#2        || 253/2000 [51:59<5:58:52, 12.33s/it]
infer : od-8220                                             |  13%|#2        || 254/2000 [52:11<5:58:35, 12.32s/it]
infer : od-8220                                             |  13%|#2        || 255/2000 [52:23<5:58:24, 12.32s/it]
infer : od-8220                                             |  13%|#2        || 256/2000 [52:36<5:58:07, 12.32s/it]
infer : od-8220                                             |  13%|#2        || 257/2000 [52:48<5:58:08, 12.33s/it]
infer : od-8220                                             |  13%|#2        || 258/2000 [53:00<5:57:44, 12.32s/it]
infer : od-8220                                             |  13%|#2        || 259/2000 [53:13<5:57:19, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 260/2000 [53:25<5:56:59, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 261/2000 [53:37<5:56:43, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 262/2000 [53:50<5:56:53, 12.32s/it]
infer : od-8220                                             |  13%|#3        || 263/2000 [54:02<5:56:37, 12.32s/it]
infer : od-8220                                             |  13%|#3        || 264/2000 [54:14<5:56:12, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 265/2000 [54:26<5:55:54, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 266/2000 [54:39<5:55:47, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 267/2000 [54:51<5:55:34, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 268/2000 [55:03<5:55:25, 12.31s/it]
infer : od-8220                                             |  13%|#3        || 269/2000 [55:16<5:55:17, 12.32s/it]
infer : od-8220                                             |  14%|#3        || 270/2000 [55:28<5:54:48, 12.31s/it]
infer : od-8220                                             |  14%|#3        || 271/2000 [55:40<5:54:45, 12.31s/it]
infer : od-8220                                             |  14%|#3        || 272/2000 [55:53<5:55:04, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 273/2000 [56:05<5:54:54, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 274/2000 [56:17<5:54:41, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 275/2000 [56:30<5:54:27, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 276/2000 [56:42<5:54:26, 12.34s/it]
infer : od-8220                                             |  14%|#3        || 277/2000 [56:54<5:53:58, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 278/2000 [57:07<5:53:44, 12.33s/it]
infer : od-8220                                             |  14%|#3        || 279/2000 [57:19<5:53:22, 12.32s/it]
infer : od-8220                                             |  14%|#4        || 280/2000 [57:31<5:53:24, 12.33s/it]
infer : od-8220                                             |  14%|#4        || 281/2000 [57:44<5:53:05, 12.32s/it]
infer : od-8220                                             |  14%|#4        || 282/2000 [57:56<5:52:42, 12.32s/it]
infer : od-8220                                             |  14%|#4        || 283/2000 [58:08<5:52:23, 12.31s/it]
infer : od-8220                                             |  14%|#4        || 284/2000 [58:21<5:52:16, 12.32s/it]
infer : od-8220                                             |  14%|#4        || 285/2000 [58:33<5:52:00, 12.32s/it]
infer : od-8220                                             |  14%|#4        || 286/2000 [58:45<5:52:07, 12.33s/it]
infer : od-8220                                             |  14%|#4        || 287/2000 [58:58<5:52:20, 12.34s/it]
infer : od-8220                                             |  14%|#4        || 288/2000 [59:10<5:51:53, 12.33s/it]
infer : od-8220                                             |  14%|#4        || 289/2000 [59:22<5:51:39, 12.33s/it]
infer : od-8220                                             |  14%|#4        || 290/2000 [59:35<5:51:41, 12.34s/it]
infer : od-8220                                             |  15%|#4        || 291/2000 [59:47<5:51:23, 12.34s/it]
infer : od-8220                                             |  15%|#4        || 292/2000 [59:59<5:51:08, 12.34s/it]
infer : od-8220                                             |  15%|#4        || 293/2000 [1:00:12<5:50:52, 12.33s/it]
infer : od-8220                                             |  15%|#4        || 294/2000 [1:00:24<5:50:39, 12.33s/it]
infer : od-8220                                             |  15%|#4        || 295/2000 [1:00:36<5:50:49, 12.35s/it]
infer : od-8220                                             |  15%|#4        || 296/2000 [1:00:49<5:50:44, 12.35s/it]
infer : od-8220                                             |  15%|#4        || 297/2000 [1:01:01<5:50:27, 12.35s/it]
infer : od-8220                                             |  15%|#4        || 298/2000 [1:01:13<5:50:12, 12.35s/it]
infer : od-8220                                             |  15%|#4        || 299/2000 [1:01:26<5:49:52, 12.34s/it]
infer : od-8220                                             |  15%|#5        || 300/2000 [1:01:38<5:49:47, 12.35s/it]
infer : od-8220                                             |  15%|#5        || 301/2000 [1:01:50<5:49:42, 12.35s/it]
infer : od-8220                                             |  15%|#5        || 302/2000 [1:02:03<5:49:43, 12.36s/it]
infer : od-8220                                             |  15%|#5        || 303/2000 [1:02:15<5:49:22, 12.35s/it]
infer : od-8220                                             |  15%|#5        || 304/2000 [1:02:27<5:48:57, 12.35s/it]
infer : od-8220                                             |  15%|#5        || 305/2000 [1:02:40<5:48:40, 12.34s/it]
infer : od-8220                                             |  15%|#5        || 306/2000 [1:02:52<5:48:19, 12.34s/it]
infer : od-8220                                             |  15%|#5        || 307/2000 [1:03:04<5:48:05, 12.34s/it]
infer : od-8220                                             |  15%|#5        || 308/2000 [1:03:17<5:48:04, 12.34s/it]
infer : od-8220                                             |  15%|#5        || 309/2000 [1:03:29<5:47:47, 12.34s/it]
infer : od-8220                                             |  16%|#5        || 310/2000 [1:03:41<5:47:25, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 311/2000 [1:03:54<5:47:02, 12.33s/it]
INFO:20250320-004316: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004416: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004516: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004616: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004716: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004816: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-004917: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005017: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005117: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005217: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005317: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005417: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005517: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005617: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005717: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005817: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-005917: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010017: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010117: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010217: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010317: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010417: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010517: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010617: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010717: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010817: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-010917: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011017: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011117: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011217: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011317: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011417: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011517: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011617: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011717: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011817: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-011917: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012017: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012117: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012217: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012317: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012417: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012517: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012617: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012717: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012817: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-012917: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013017: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013117: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013218: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013318: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013418: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013518: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013618: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013718: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013818: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-013918: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014018: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014118: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014218: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014318: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014418: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014518: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  16%|#5        || 312/2000 [1:04:06<5:46:47, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 313/2000 [1:04:18<5:46:31, 12.32s/it]
infer : od-8220                                             |  16%|#5        || 314/2000 [1:04:31<5:46:23, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 315/2000 [1:04:43<5:46:14, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 316/2000 [1:04:55<5:46:00, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 317/2000 [1:05:08<5:45:43, 12.33s/it]
infer : od-8220                                             |  16%|#5        || 318/2000 [1:05:20<5:45:27, 12.32s/it]
infer : od-8220                                             |  16%|#5        || 319/2000 [1:05:32<5:45:14, 12.32s/it]
infer : od-8220                                             |  16%|#6        || 320/2000 [1:05:45<5:45:16, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 321/2000 [1:05:57<5:44:54, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 322/2000 [1:06:09<5:44:36, 12.32s/it]
infer : od-8220                                             |  16%|#6        || 323/2000 [1:06:22<5:44:25, 12.32s/it]
infer : od-8220                                             |  16%|#6        || 324/2000 [1:06:34<5:44:16, 12.32s/it]
infer : od-8220                                             |  16%|#6        || 325/2000 [1:06:46<5:44:13, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 326/2000 [1:06:59<5:44:07, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 327/2000 [1:07:11<5:43:49, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 328/2000 [1:07:23<5:43:31, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 329/2000 [1:07:36<5:43:17, 12.33s/it]
infer : od-8220                                             |  16%|#6        || 330/2000 [1:07:48<5:42:53, 12.32s/it]
infer : od-8220                                             |  17%|#6        || 331/2000 [1:08:00<5:42:46, 12.32s/it]
infer : od-8220                                             |  17%|#6        || 332/2000 [1:08:13<5:42:39, 12.33s/it]
infer : od-8220                                             |  17%|#6        || 333/2000 [1:08:25<5:42:30, 12.33s/it]
infer : od-8220                                             |  17%|#6        || 334/2000 [1:08:37<5:42:15, 12.33s/it]
infer : od-8220                                             |  17%|#6        || 335/2000 [1:08:50<5:42:00, 12.32s/it]
infer : od-8220                                             |  17%|#6        || 336/2000 [1:09:02<5:41:50, 12.33s/it]
infer : od-8220                                             |  17%|#6        || 337/2000 [1:09:14<5:41:33, 12.32s/it]
infer : od-8220                                             |  17%|#6        || 338/2000 [1:09:27<5:41:24, 12.33s/it]
infer : od-8220                                             |  17%|#6        || 339/2000 [1:09:39<5:41:09, 12.32s/it]
infer : od-8220                                             |  17%|#7        || 340/2000 [1:09:51<5:40:56, 12.32s/it]
infer : od-8220                                             |  17%|#7        || 341/2000 [1:10:04<5:40:50, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 342/2000 [1:10:16<5:40:39, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 343/2000 [1:10:28<5:40:35, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 344/2000 [1:10:41<5:40:21, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 345/2000 [1:10:53<5:40:05, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 346/2000 [1:11:05<5:39:44, 12.32s/it]
infer : od-8220                                             |  17%|#7        || 347/2000 [1:11:17<5:39:35, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 348/2000 [1:11:30<5:39:29, 12.33s/it]
infer : od-8220                                             |  17%|#7        || 349/2000 [1:11:42<5:39:33, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 350/2000 [1:11:55<5:39:19, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 351/2000 [1:12:07<5:39:07, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 352/2000 [1:12:19<5:38:47, 12.33s/it]
infer : od-8220                                             |  18%|#7        || 353/2000 [1:12:32<5:38:39, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 354/2000 [1:12:44<5:38:28, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 355/2000 [1:12:56<5:38:20, 12.34s/it]
infer : od-8220                                             |  18%|#7        || 356/2000 [1:13:09<5:37:54, 12.33s/it]
infer : od-8220                                             |  18%|#7        || 357/2000 [1:13:21<5:37:32, 12.33s/it]
infer : od-8220                                             |  18%|#7        || 358/2000 [1:13:33<5:37:31, 12.33s/it]
infer : od-8220                                             |  18%|#7        || 359/2000 [1:13:46<5:37:26, 12.34s/it]
infer : od-8220                                             |  18%|#8        || 360/2000 [1:13:58<5:37:07, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 361/2000 [1:14:10<5:36:54, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 362/2000 [1:14:23<5:36:32, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 363/2000 [1:14:35<5:36:26, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 364/2000 [1:14:47<5:36:22, 12.34s/it]
infer : od-8220                                             |  18%|#8        || 365/2000 [1:15:00<5:36:12, 12.34s/it]
infer : od-8220                                             |  18%|#8        || 366/2000 [1:15:12<5:36:04, 12.34s/it]
infer : od-8220                                             |  18%|#8        || 367/2000 [1:15:24<5:35:44, 12.34s/it]
infer : od-8220                                             |  18%|#8        || 368/2000 [1:15:37<5:35:24, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 369/2000 [1:15:49<5:35:12, 12.33s/it]
infer : od-8220                                             |  18%|#8        || 370/2000 [1:16:01<5:34:54, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 371/2000 [1:16:14<5:34:46, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 372/2000 [1:16:26<5:34:36, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 373/2000 [1:16:38<5:34:20, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 374/2000 [1:16:51<5:34:05, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 375/2000 [1:17:03<5:33:57, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 376/2000 [1:17:15<5:33:43, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 377/2000 [1:17:28<5:33:33, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 378/2000 [1:17:40<5:33:24, 12.33s/it]
infer : od-8220                                             |  19%|#8        || 379/2000 [1:17:52<5:33:03, 12.33s/it]
infer : od-8220                                             |  19%|#9        || 380/2000 [1:18:04<5:32:43, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 381/2000 [1:18:17<5:32:25, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 382/2000 [1:18:29<5:32:19, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 383/2000 [1:18:41<5:31:57, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 384/2000 [1:18:54<5:31:56, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 385/2000 [1:19:06<5:31:42, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 386/2000 [1:19:18<5:31:30, 12.32s/it]
infer : od-8220                                             |  19%|#9        || 387/2000 [1:19:31<5:31:30, 12.33s/it]
infer : od-8220                                             |  19%|#9        || 388/2000 [1:19:43<5:31:19, 12.33s/it]
infer : od-8220                                             |  19%|#9        || 389/2000 [1:19:55<5:31:08, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 390/2000 [1:20:08<5:30:50, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 391/2000 [1:20:20<5:30:26, 12.32s/it]
infer : od-8220                                             |  20%|#9        || 392/2000 [1:20:32<5:30:22, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 393/2000 [1:20:45<5:30:14, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 394/2000 [1:20:57<5:30:12, 12.34s/it]
infer : od-8220                                             |  20%|#9        || 395/2000 [1:21:09<5:29:56, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 396/2000 [1:21:22<5:29:47, 12.34s/it]
infer : od-8220                                             |  20%|#9        || 397/2000 [1:21:34<5:29:32, 12.33s/it]
infer : od-8220                                             |  20%|#9        || 398/2000 [1:21:46<5:29:23, 12.34s/it]
infer : od-8220                                             |  20%|#9        || 399/2000 [1:21:59<5:29:10, 12.34s/it]
infer : od-8220                                             |  20%|##        || 400/2000 [1:22:11<5:29:04, 12.34s/it]
infer : od-8220                                             |  20%|##        || 401/2000 [1:22:23<5:28:36, 12.33s/it]
infer : od-8220                                             |  20%|##        || 402/2000 [1:22:36<5:28:28, 12.33s/it]
infer : od-8220                                             |  20%|##        || 403/2000 [1:22:48<5:28:10, 12.33s/it]
infer : od-8220                                             |  20%|##        || 404/2000 [1:23:00<5:28:00, 12.33s/it]
infer : od-8220                                             |  20%|##        || 405/2000 [1:23:13<5:27:51, 12.33s/it]
infer : od-8220                                             |  20%|##        || 406/2000 [1:23:25<5:27:39, 12.33s/it]
infer : od-8220                                             |  20%|##        || 407/2000 [1:23:37<5:27:25, 12.33s/it]
infer : od-8220                                             |  20%|##        || 408/2000 [1:23:50<5:27:10, 12.33s/it]
infer : od-8220                                             |  20%|##        || 409/2000 [1:24:02<5:26:50, 12.33s/it]
infer : od-8220                                             |  20%|##        || 410/2000 [1:24:14<5:26:45, 12.33s/it]
infer : od-8220                                             |  21%|##        || 411/2000 [1:24:27<5:26:34, 12.33s/it]
infer : od-8220                                             |  21%|##        || 412/2000 [1:24:39<5:26:14, 12.33s/it]
infer : od-8220                                             |  21%|##        || 413/2000 [1:24:51<5:26:02, 12.33s/it]
infer : od-8220                                             |  21%|##        || 414/2000 [1:25:04<5:25:49, 12.33s/it]
infer : od-8220                                             |  21%|##        || 415/2000 [1:25:16<5:25:41, 12.33s/it]
infer : od-8220                                             |  21%|##        || 416/2000 [1:25:28<5:25:36, 12.33s/it]
infer : od-8220                                             |  21%|##        || 417/2000 [1:25:41<5:25:31, 12.34s/it]
infer : od-8220                                             |  21%|##        || 418/2000 [1:25:53<5:25:17, 12.34s/it]
infer : od-8220                                             |  21%|##        || 419/2000 [1:26:05<5:25:04, 12.34s/it]
infer : od-8220                                             |  21%|##1       || 420/2000 [1:26:18<5:24:53, 12.34s/it]
infer : od-8220                                             |  21%|##1       || 421/2000 [1:26:30<5:24:40, 12.34s/it]
infer : od-8220                                             |  21%|##1       || 422/2000 [1:26:42<5:24:24, 12.33s/it]
infer : od-8220                                             |  21%|##1       || 423/2000 [1:26:55<5:24:09, 12.33s/it]
infer : od-8220                                             |  21%|##1       || 424/2000 [1:27:07<5:23:50, 12.33s/it]
infer : od-8220                                             |  21%|##1       || 425/2000 [1:27:19<5:23:25, 12.32s/it]
infer : od-8220                                             |  21%|##1       || 426/2000 [1:27:32<5:23:26, 12.33s/it]
infer : od-8220                                             |  21%|##1       || 427/2000 [1:27:44<5:23:26, 12.34s/it]
infer : od-8220                                             |  21%|##1       || 428/2000 [1:27:56<5:23:15, 12.34s/it]
infer : od-8220                                             |  21%|##1       || 429/2000 [1:28:09<5:22:57, 12.33s/it]
infer : od-8220                                             |  22%|##1       || 430/2000 [1:28:21<5:22:40, 12.33s/it]
infer : od-8220                                             |  22%|##1       || 431/2000 [1:28:33<5:22:16, 12.32s/it]
infer : od-8220                                             |  22%|##1       || 432/2000 [1:28:46<5:22:19, 12.33s/it]
infer : od-8220                                             |  22%|##1       || 433/2000 [1:28:58<5:22:04, 12.33s/it]
infer : od-8220                                             |  22%|##1       || 434/2000 [1:29:10<5:21:35, 12.32s/it]
infer : od-8220                                             |  22%|##1       || 435/2000 [1:29:23<5:21:17, 12.32s/it]
infer : od-8220                                             |  22%|##1       || 436/2000 [1:29:35<5:21:01, 12.32s/it]
infer : od-8220                                             |  22%|##1       || 437/2000 [1:29:47<5:20:47, 12.31s/it]
infer : od-8220                                             |  22%|##1       || 438/2000 [1:30:00<5:20:43, 12.32s/it]
infer : od-8220                                             |  22%|##1       || 439/2000 [1:30:12<5:20:40, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 440/2000 [1:30:24<5:20:32, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 441/2000 [1:30:37<5:20:21, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 442/2000 [1:30:49<5:20:05, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 443/2000 [1:31:01<5:20:04, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 444/2000 [1:31:14<5:19:56, 12.34s/it]
infer : od-8220                                             |  22%|##2       || 445/2000 [1:31:26<5:19:46, 12.34s/it]
infer : od-8220                                             |  22%|##2       || 446/2000 [1:31:38<5:19:34, 12.34s/it]
infer : od-8220                                             |  22%|##2       || 447/2000 [1:31:51<5:19:13, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 448/2000 [1:32:03<5:18:48, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 449/2000 [1:32:15<5:18:48, 12.33s/it]
infer : od-8220                                             |  22%|##2       || 450/2000 [1:32:28<5:18:40, 12.34s/it]
infer : od-8220                                             |  23%|##2       || 451/2000 [1:32:40<5:18:29, 12.34s/it]
infer : od-8220                                             |  23%|##2       || 452/2000 [1:32:52<5:18:21, 12.34s/it]
infer : od-8220                                             |  23%|##2       || 453/2000 [1:33:05<5:18:02, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 454/2000 [1:33:17<5:17:43, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 455/2000 [1:33:29<5:17:31, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 456/2000 [1:33:42<5:17:23, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 457/2000 [1:33:54<5:17:12, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 458/2000 [1:34:06<5:16:51, 12.33s/it]
infer : od-8220                                             |  23%|##2       || 459/2000 [1:34:19<5:16:32, 12.32s/it]
infer : od-8220                                             |  23%|##3       || 460/2000 [1:34:31<5:16:20, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 461/2000 [1:34:43<5:16:19, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 462/2000 [1:34:56<5:16:15, 12.34s/it]
infer : od-8220                                             |  23%|##3       || 463/2000 [1:35:08<5:15:56, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 464/2000 [1:35:20<5:15:39, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 465/2000 [1:35:33<5:15:21, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 466/2000 [1:35:45<5:15:22, 12.34s/it]
infer : od-8220                                             |  23%|##3       || 467/2000 [1:35:57<5:15:15, 12.34s/it]
infer : od-8220                                             |  23%|##3       || 468/2000 [1:36:10<5:14:50, 12.33s/it]
infer : od-8220                                             |  23%|##3       || 469/2000 [1:36:22<5:14:30, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 470/2000 [1:36:34<5:14:22, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 471/2000 [1:36:47<5:14:09, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 472/2000 [1:36:59<5:13:58, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 473/2000 [1:37:11<5:13:54, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 474/2000 [1:37:24<5:13:40, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 475/2000 [1:37:36<5:13:32, 12.34s/it]
infer : od-8220                                             |  24%|##3       || 476/2000 [1:37:48<5:13:17, 12.33s/it]
infer : od-8220                                             |  24%|##3       || 477/2000 [1:38:01<5:13:12, 12.34s/it]
infer : od-8220                                             |  24%|##3       || 478/2000 [1:38:13<5:12:54, 12.34s/it]
infer : od-8220                                             |  24%|##3       || 479/2000 [1:38:25<5:12:42, 12.34s/it]
infer : od-8220                                             |  24%|##4       || 480/2000 [1:38:38<5:12:28, 12.33s/it]
infer : od-8220                                             |  24%|##4       || 481/2000 [1:38:50<5:12:16, 12.33s/it]
infer : od-8220                                             |  24%|##4       || 482/2000 [1:39:02<5:12:05, 12.34s/it]
infer : od-8220                                             |  24%|##4       || 483/2000 [1:39:15<5:11:58, 12.34s/it]
infer : od-8220                                             |  24%|##4       || 484/2000 [1:39:27<5:11:42, 12.34s/it]
infer : od-8220                                             |  24%|##4       || 485/2000 [1:39:39<5:11:28, 12.34s/it]
infer : od-8220                                             |  24%|##4       || 486/2000 [1:39:52<5:11:08, 12.33s/it]
infer : od-8220                                             |  24%|##4       || 487/2000 [1:40:04<5:10:44, 12.32s/it]
infer : od-8220                                             |  24%|##4       || 488/2000 [1:40:16<5:10:41, 12.33s/it]
infer : od-8220                                             |  24%|##4       || 489/2000 [1:40:29<5:10:34, 12.33s/it]
infer : od-8220                                             |  24%|##4       || 490/2000 [1:40:41<5:10:20, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 491/2000 [1:40:53<5:10:16, 12.34s/it]
infer : od-8220                                             |  25%|##4       || 492/2000 [1:41:06<5:09:54, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 493/2000 [1:41:18<5:09:36, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 494/2000 [1:41:30<5:09:27, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 495/2000 [1:41:43<5:09:15, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 496/2000 [1:41:55<5:09:09, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 497/2000 [1:42:07<5:08:55, 12.33s/it]
infer : od-8220                                             |  25%|##4       || 498/2000 [1:42:20<5:08:30, 12.32s/it]
infer : od-8220                                             |  25%|##4       || 499/2000 [1:42:32<5:08:18, 12.32s/it]
infer : od-8220                                             |  25%|##5       || 500/2000 [1:42:44<5:08:16, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 501/2000 [1:42:57<5:08:06, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 502/2000 [1:43:09<5:07:48, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 503/2000 [1:43:21<5:07:25, 12.32s/it]
infer : od-8220                                             |  25%|##5       || 504/2000 [1:43:33<5:07:13, 12.32s/it]
infer : od-8220                                             |  25%|##5       || 505/2000 [1:43:46<5:07:07, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 506/2000 [1:43:58<5:07:00, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 507/2000 [1:44:10<5:06:43, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 508/2000 [1:44:23<5:06:33, 12.33s/it]
infer : od-8220                                             |  25%|##5       || 509/2000 [1:44:35<5:06:17, 12.33s/it]
infer : od-8220                                             |  26%|##5       || 510/2000 [1:44:47<5:06:03, 12.32s/it]
infer : od-8220                                             |  26%|##5       || 511/2000 [1:45:00<5:05:45, 12.32s/it]
infer : od-8220                                             |  26%|##5       || 512/2000 [1:45:12<5:05:24, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 513/2000 [1:45:24<5:04:58, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 514/2000 [1:45:37<5:04:48, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 515/2000 [1:45:49<5:04:37, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 516/2000 [1:46:01<5:04:27, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 517/2000 [1:46:14<5:04:21, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 518/2000 [1:46:26<5:04:06, 12.31s/it]
infer : od-8220                                             |  26%|##5       || 519/2000 [1:46:38<5:03:53, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 520/2000 [1:46:51<5:03:29, 12.30s/it]
infer : od-8220                                             |  26%|##6       || 521/2000 [1:47:03<5:03:15, 12.30s/it]
infer : od-8220                                             |  26%|##6       || 522/2000 [1:47:15<5:03:11, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 523/2000 [1:47:27<5:02:59, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 524/2000 [1:47:40<5:02:48, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 525/2000 [1:47:52<5:02:28, 12.30s/it]
infer : od-8220                                             |  26%|##6       || 526/2000 [1:48:04<5:02:13, 12.30s/it]
infer : od-8220                                             |  26%|##6       || 527/2000 [1:48:17<5:02:08, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 528/2000 [1:48:29<5:02:03, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 529/2000 [1:48:41<5:01:44, 12.31s/it]
infer : od-8220                                             |  26%|##6       || 530/2000 [1:48:54<5:01:30, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 531/2000 [1:49:06<5:01:19, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 532/2000 [1:49:18<5:01:12, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 533/2000 [1:49:31<5:01:01, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 534/2000 [1:49:43<5:00:56, 12.32s/it]
infer : od-8220                                             |  27%|##6       || 535/2000 [1:49:55<5:00:36, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 536/2000 [1:50:07<5:00:14, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 537/2000 [1:50:20<4:59:59, 12.30s/it]
infer : od-8220                                             |  27%|##6       || 538/2000 [1:50:32<4:59:50, 12.31s/it]
infer : od-8220                                             |  27%|##6       || 539/2000 [1:50:44<4:59:49, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 540/2000 [1:50:57<4:59:44, 12.32s/it]
infer : od-8220                                             |  27%|##7       || 541/2000 [1:51:09<4:59:22, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 542/2000 [1:51:21<4:59:03, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 543/2000 [1:51:34<4:58:38, 12.30s/it]
infer : od-8220                                             |  27%|##7       || 544/2000 [1:51:46<4:58:39, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 545/2000 [1:51:58<4:58:32, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 546/2000 [1:52:11<4:58:22, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 547/2000 [1:52:23<4:57:59, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 548/2000 [1:52:35<4:57:47, 12.31s/it]
infer : od-8220                                             |  27%|##7       || 549/2000 [1:52:47<4:57:40, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 550/2000 [1:53:00<4:57:25, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 551/2000 [1:53:12<4:57:16, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 552/2000 [1:53:24<4:57:03, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 553/2000 [1:53:37<4:56:47, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 554/2000 [1:53:49<4:56:33, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 555/2000 [1:54:01<4:56:23, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 556/2000 [1:54:14<4:56:11, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 557/2000 [1:54:26<4:55:58, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 558/2000 [1:54:38<4:55:44, 12.31s/it]
infer : od-8220                                             |  28%|##7       || 559/2000 [1:54:51<4:55:27, 12.30s/it]
infer : od-8220                                             |  28%|##8       || 560/2000 [1:55:03<4:55:12, 12.30s/it]
infer : od-8220                                             |  28%|##8       || 561/2000 [1:55:15<4:55:07, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 562/2000 [1:55:27<4:54:51, 12.30s/it]
infer : od-8220                                             |  28%|##8       || 563/2000 [1:55:40<4:54:48, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 564/2000 [1:55:52<4:54:40, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 565/2000 [1:56:04<4:54:22, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 566/2000 [1:56:17<4:54:12, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 567/2000 [1:56:29<4:53:57, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 568/2000 [1:56:41<4:53:41, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 569/2000 [1:56:54<4:53:31, 12.31s/it]
infer : od-8220                                             |  28%|##8       || 570/2000 [1:57:06<4:53:11, 12.30s/it]
infer : od-8220                                             |  29%|##8       || 571/2000 [1:57:18<4:52:59, 12.30s/it]
infer : od-8220                                             |  29%|##8       || 572/2000 [1:57:31<4:52:53, 12.31s/it]
infer : od-8220                                             |  29%|##8       || 573/2000 [1:57:43<4:52:54, 12.32s/it]
infer : od-8220                                             |  29%|##8       || 574/2000 [1:57:55<4:52:43, 12.32s/it]
infer : od-8220                                             |  29%|##8       || 575/2000 [1:58:07<4:52:23, 12.31s/it]
infer : od-8220                                             |  29%|##8       || 576/2000 [1:58:20<4:51:58, 12.30s/it]
infer : od-8220                                             |  29%|##8       || 577/2000 [1:58:32<4:51:43, 12.30s/it]
infer : od-8220                                             |  29%|##8       || 578/2000 [1:58:44<4:51:49, 12.31s/it]
infer : od-8220                                             |  29%|##8       || 579/2000 [1:58:57<4:51:31, 12.31s/it]
infer : od-8220                                             |  29%|##9       || 580/2000 [1:59:09<4:51:13, 12.31s/it]
infer : od-8220                                             |  29%|##9       || 581/2000 [1:59:21<4:50:59, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 582/2000 [1:59:34<4:50:42, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 583/2000 [1:59:46<4:50:32, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 584/2000 [1:59:58<4:50:22, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 585/2000 [2:00:11<4:50:16, 12.31s/it]
infer : od-8220                                             |  29%|##9       || 586/2000 [2:00:23<4:50:01, 12.31s/it]
infer : od-8220                                             |  29%|##9       || 587/2000 [2:00:35<4:49:42, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 588/2000 [2:00:47<4:49:31, 12.30s/it]
infer : od-8220                                             |  29%|##9       || 589/2000 [2:01:00<4:49:29, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 590/2000 [2:01:12<4:49:21, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 591/2000 [2:01:24<4:49:09, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 592/2000 [2:01:37<4:48:58, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 593/2000 [2:01:49<4:48:35, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 594/2000 [2:02:01<4:48:22, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 595/2000 [2:02:14<4:48:15, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 596/2000 [2:02:26<4:48:01, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 597/2000 [2:02:38<4:47:55, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 598/2000 [2:02:51<4:47:38, 12.31s/it]
infer : od-8220                                             |  30%|##9       || 599/2000 [2:03:03<4:47:23, 12.31s/it]
infer : od-8220                                             |  30%|###       || 600/2000 [2:03:15<4:47:11, 12.31s/it]
infer : od-8220                                             |  30%|###       || 601/2000 [2:03:27<4:46:58, 12.31s/it]
infer : od-8220                                             |  30%|###       || 602/2000 [2:03:40<4:46:48, 12.31s/it]
infer : od-8220                                             |  30%|###       || 603/2000 [2:03:52<4:46:30, 12.31s/it]
infer : od-8220                                             |  30%|###       || 604/2000 [2:04:04<4:46:17, 12.30s/it]
infer : od-8220                                             |  30%|###       || 605/2000 [2:04:17<4:46:06, 12.31s/it]
infer : od-8220                                             |  30%|###       || 606/2000 [2:04:29<4:45:59, 12.31s/it]
infer : od-8220                                             |  30%|###       || 607/2000 [2:04:41<4:45:46, 12.31s/it]
infer : od-8220                                             |  30%|###       || 608/2000 [2:04:54<4:45:31, 12.31s/it]
infer : od-8220                                             |  30%|###       || 609/2000 [2:05:06<4:45:09, 12.30s/it]
infer : od-8220                                             |  30%|###       || 610/2000 [2:05:18<4:44:53, 12.30s/it]
infer : od-8220                                             |  31%|###       || 611/2000 [2:05:30<4:44:44, 12.30s/it]
infer : od-8220                                             |  31%|###       || 612/2000 [2:05:43<4:44:46, 12.31s/it]
infer : od-8220                                             |  31%|###       || 613/2000 [2:05:55<4:44:24, 12.30s/it]
infer : od-8220                                             |  31%|###       || 614/2000 [2:06:07<4:44:04, 12.30s/it]
infer : od-8220                                             |  31%|###       || 615/2000 [2:06:20<4:43:48, 12.29s/it]
infer : od-8220                                             |  31%|###       || 616/2000 [2:06:32<4:43:37, 12.30s/it]
infer : od-8220                                             |  31%|###       || 617/2000 [2:06:44<4:43:32, 12.30s/it]
infer : od-8220                                             |  31%|###       || 618/2000 [2:06:57<4:43:23, 12.30s/it]
INFO:20250320-014618: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014718: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014818: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-014918: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015018: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015118: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015218: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015318: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015418: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015518: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015618: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015718: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015818: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-015918: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020018: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020118: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020218: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020318: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020418: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020518: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020618: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020718: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020818: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-020918: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021018: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021118: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021218: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021318: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021418: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021518: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021619: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021719: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021819: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-021919: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022019: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022119: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022219: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022319: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022419: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022519: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022619: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022719: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022819: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-022919: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023019: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023119: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023219: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023319: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023419: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023519: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023619: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023719: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023819: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-023919: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024019: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024119: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024219: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024319: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024419: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024519: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024619: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024719: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-024819: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  31%|###       || 619/2000 [2:07:09<4:43:12, 12.30s/it]
infer : od-8220                                             |  31%|###1      || 620/2000 [2:07:21<4:43:00, 12.31s/it]
infer : od-8220                                             |  31%|###1      || 621/2000 [2:07:33<4:42:40, 12.30s/it]
infer : od-8220                                             |  31%|###1      || 622/2000 [2:07:46<4:42:38, 12.31s/it]
infer : od-8220                                             |  31%|###1      || 623/2000 [2:07:58<4:42:29, 12.31s/it]
infer : od-8220                                             |  31%|###1      || 624/2000 [2:08:10<4:42:16, 12.31s/it]
infer : od-8220                                             |  31%|###1      || 625/2000 [2:08:23<4:41:59, 12.31s/it]
infer : od-8220                                             |  31%|###1      || 626/2000 [2:08:35<4:41:41, 12.30s/it]
infer : od-8220                                             |  31%|###1      || 627/2000 [2:08:47<4:41:26, 12.30s/it]
infer : od-8220                                             |  31%|###1      || 628/2000 [2:09:00<4:41:15, 12.30s/it]
infer : od-8220                                             |  31%|###1      || 629/2000 [2:09:12<4:41:03, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 630/2000 [2:09:24<4:40:53, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 631/2000 [2:09:37<4:40:46, 12.31s/it]
infer : od-8220                                             |  32%|###1      || 632/2000 [2:09:49<4:40:26, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 633/2000 [2:10:01<4:40:15, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 634/2000 [2:10:13<4:40:05, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 635/2000 [2:10:26<4:39:55, 12.30s/it]
infer : od-8220                                             |  32%|###1      || 636/2000 [2:10:38<4:39:48, 12.31s/it]
infer : od-8220                                             |  32%|###1      || 637/2000 [2:10:50<4:39:32, 12.31s/it]
infer : od-8220                                             |  32%|###1      || 638/2000 [2:11:03<4:39:22, 12.31s/it]
infer : od-8220                                             |  32%|###1      || 639/2000 [2:11:15<4:39:07, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 640/2000 [2:11:27<4:39:01, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 641/2000 [2:11:40<4:38:49, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 642/2000 [2:11:52<4:38:32, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 643/2000 [2:12:04<4:38:19, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 644/2000 [2:12:17<4:38:11, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 645/2000 [2:12:29<4:38:00, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 646/2000 [2:12:41<4:37:55, 12.32s/it]
infer : od-8220                                             |  32%|###2      || 647/2000 [2:12:53<4:37:41, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 648/2000 [2:13:06<4:37:22, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 649/2000 [2:13:18<4:37:12, 12.31s/it]
infer : od-8220                                             |  32%|###2      || 650/2000 [2:13:30<4:36:59, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 651/2000 [2:13:43<4:36:55, 12.32s/it]
infer : od-8220                                             |  33%|###2      || 652/2000 [2:13:55<4:36:46, 12.32s/it]
infer : od-8220                                             |  33%|###2      || 653/2000 [2:14:07<4:36:25, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 654/2000 [2:14:20<4:36:08, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 655/2000 [2:14:32<4:35:51, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 656/2000 [2:14:44<4:35:48, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 657/2000 [2:14:57<4:35:32, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 658/2000 [2:15:09<4:35:18, 12.31s/it]
infer : od-8220                                             |  33%|###2      || 659/2000 [2:15:21<4:35:01, 12.31s/it]
infer : od-8220                                             |  33%|###3      || 660/2000 [2:15:34<4:34:51, 12.31s/it]
infer : od-8220                                             |  33%|###3      || 661/2000 [2:15:46<4:34:52, 12.32s/it]
infer : od-8220                                             |  33%|###3      || 662/2000 [2:15:58<4:34:42, 12.32s/it]
infer : od-8220                                             |  33%|###3      || 663/2000 [2:16:10<4:34:26, 12.32s/it]
infer : od-8220                                             |  33%|###3      || 664/2000 [2:16:23<4:34:10, 12.31s/it]
infer : od-8220                                             |  33%|###3      || 665/2000 [2:16:35<4:33:57, 12.31s/it]
infer : od-8220                                             |  33%|###3      || 666/2000 [2:16:47<4:33:44, 12.31s/it]
infer : od-8220                                             |  33%|###3      || 667/2000 [2:17:00<4:33:41, 12.32s/it]
infer : od-8220                                             |  33%|###3      || 668/2000 [2:17:12<4:33:30, 12.32s/it]
infer : od-8220                                             |  33%|###3      || 669/2000 [2:17:24<4:33:14, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 670/2000 [2:17:37<4:33:07, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 671/2000 [2:17:49<4:32:57, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 672/2000 [2:18:01<4:32:49, 12.33s/it]
infer : od-8220                                             |  34%|###3      || 673/2000 [2:18:14<4:32:36, 12.33s/it]
infer : od-8220                                             |  34%|###3      || 674/2000 [2:18:26<4:32:20, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 675/2000 [2:18:38<4:31:57, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 676/2000 [2:18:51<4:31:44, 12.31s/it]
infer : od-8220                                             |  34%|###3      || 677/2000 [2:19:03<4:31:32, 12.31s/it]
infer : od-8220                                             |  34%|###3      || 678/2000 [2:19:15<4:31:23, 12.32s/it]
infer : od-8220                                             |  34%|###3      || 679/2000 [2:19:28<4:31:11, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 680/2000 [2:19:40<4:31:05, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 681/2000 [2:19:52<4:30:49, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 682/2000 [2:20:05<4:30:33, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 683/2000 [2:20:17<4:30:19, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 684/2000 [2:20:29<4:30:07, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 685/2000 [2:20:41<4:29:57, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 686/2000 [2:20:54<4:29:48, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 687/2000 [2:21:06<4:29:31, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 688/2000 [2:21:18<4:29:18, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 689/2000 [2:21:31<4:29:13, 12.32s/it]
infer : od-8220                                             |  34%|###4      || 690/2000 [2:21:43<4:29:05, 12.32s/it]
infer : od-8220                                             |  35%|###4      || 691/2000 [2:21:55<4:28:40, 12.32s/it]
infer : od-8220                                             |  35%|###4      || 692/2000 [2:22:08<4:28:20, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 693/2000 [2:22:20<4:27:59, 12.30s/it]
infer : od-8220                                             |  35%|###4      || 694/2000 [2:22:32<4:27:57, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 695/2000 [2:22:45<4:27:45, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 696/2000 [2:22:57<4:27:34, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 697/2000 [2:23:09<4:27:24, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 698/2000 [2:23:22<4:27:07, 12.31s/it]
infer : od-8220                                             |  35%|###4      || 699/2000 [2:23:34<4:26:54, 12.31s/it]
infer : od-8220                                             |  35%|###5      || 700/2000 [2:23:46<4:26:54, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 701/2000 [2:23:59<4:26:44, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 702/2000 [2:24:11<4:26:30, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 703/2000 [2:24:23<4:26:18, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 704/2000 [2:24:35<4:26:06, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 705/2000 [2:24:48<4:25:56, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 706/2000 [2:25:00<4:25:44, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 707/2000 [2:25:12<4:25:28, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 708/2000 [2:25:25<4:25:17, 12.32s/it]
infer : od-8220                                             |  35%|###5      || 709/2000 [2:25:37<4:25:00, 12.32s/it]
infer : od-8220                                             |  36%|###5      || 710/2000 [2:25:49<4:24:46, 12.32s/it]
infer : od-8220                                             |  36%|###5      || 711/2000 [2:26:02<4:24:35, 12.32s/it]
infer : od-8220                                             |  36%|###5      || 712/2000 [2:26:14<4:24:21, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 713/2000 [2:26:26<4:24:04, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 714/2000 [2:26:39<4:23:52, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 715/2000 [2:26:51<4:23:34, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 716/2000 [2:27:03<4:23:23, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 717/2000 [2:27:16<4:23:15, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 718/2000 [2:27:28<4:23:06, 12.31s/it]
infer : od-8220                                             |  36%|###5      || 719/2000 [2:27:40<4:22:53, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 720/2000 [2:27:52<4:22:31, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 721/2000 [2:28:05<4:22:19, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 722/2000 [2:28:17<4:22:14, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 723/2000 [2:28:29<4:22:01, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 724/2000 [2:28:42<4:21:50, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 725/2000 [2:28:54<4:21:36, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 726/2000 [2:29:06<4:21:21, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 727/2000 [2:29:19<4:21:10, 12.31s/it]
infer : od-8220                                             |  36%|###6      || 728/2000 [2:29:31<4:21:05, 12.32s/it]
infer : od-8220                                             |  36%|###6      || 729/2000 [2:29:43<4:20:56, 12.32s/it]
infer : od-8220                                             |  36%|###6      || 730/2000 [2:29:56<4:20:47, 12.32s/it]
infer : od-8220                                             |  37%|###6      || 731/2000 [2:30:08<4:20:27, 12.32s/it]
infer : od-8220                                             |  37%|###6      || 732/2000 [2:30:20<4:20:08, 12.31s/it]
infer : od-8220                                             |  37%|###6      || 733/2000 [2:30:33<4:19:54, 12.31s/it]
infer : od-8220                                             |  37%|###6      || 734/2000 [2:30:45<4:19:45, 12.31s/it]
infer : od-8220                                             |  37%|###6      || 735/2000 [2:30:57<4:19:45, 12.32s/it]
infer : od-8220                                             |  37%|###6      || 736/2000 [2:31:10<4:19:26, 12.32s/it]
infer : od-8220                                             |  37%|###6      || 737/2000 [2:31:22<4:19:05, 12.31s/it]
infer : od-8220                                             |  37%|###6      || 738/2000 [2:31:34<4:18:50, 12.31s/it]
infer : od-8220                                             |  37%|###6      || 739/2000 [2:31:46<4:18:37, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 740/2000 [2:31:59<4:18:29, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 741/2000 [2:32:11<4:18:14, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 742/2000 [2:32:23<4:17:59, 12.30s/it]
infer : od-8220                                             |  37%|###7      || 743/2000 [2:32:36<4:17:45, 12.30s/it]
infer : od-8220                                             |  37%|###7      || 744/2000 [2:32:48<4:17:45, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 745/2000 [2:33:00<4:17:38, 12.32s/it]
infer : od-8220                                             |  37%|###7      || 746/2000 [2:33:13<4:17:25, 12.32s/it]
infer : od-8220                                             |  37%|###7      || 747/2000 [2:33:25<4:17:01, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 748/2000 [2:33:37<4:16:50, 12.31s/it]
infer : od-8220                                             |  37%|###7      || 749/2000 [2:33:49<4:16:35, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 750/2000 [2:34:02<4:16:24, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 751/2000 [2:34:14<4:16:12, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 752/2000 [2:34:26<4:16:02, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 753/2000 [2:34:39<4:15:43, 12.30s/it]
infer : od-8220                                             |  38%|###7      || 754/2000 [2:34:51<4:15:30, 12.30s/it]
infer : od-8220                                             |  38%|###7      || 755/2000 [2:35:03<4:15:22, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 756/2000 [2:35:16<4:15:16, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 757/2000 [2:35:28<4:15:05, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 758/2000 [2:35:40<4:14:51, 12.31s/it]
infer : od-8220                                             |  38%|###7      || 759/2000 [2:35:53<4:14:36, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 760/2000 [2:36:05<4:14:28, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 761/2000 [2:36:17<4:14:17, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 762/2000 [2:36:30<4:14:08, 12.32s/it]
infer : od-8220                                             |  38%|###8      || 763/2000 [2:36:42<4:13:54, 12.32s/it]
infer : od-8220                                             |  38%|###8      || 764/2000 [2:36:54<4:13:41, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 765/2000 [2:37:06<4:13:17, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 766/2000 [2:37:19<4:13:05, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 767/2000 [2:37:31<4:13:01, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 768/2000 [2:37:43<4:12:51, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 769/2000 [2:37:56<4:12:33, 12.31s/it]
infer : od-8220                                             |  38%|###8      || 770/2000 [2:38:08<4:12:15, 12.31s/it]
infer : od-8220                                             |  39%|###8      || 771/2000 [2:38:20<4:12:07, 12.31s/it]
infer : od-8220                                             |  39%|###8      || 772/2000 [2:38:33<4:11:52, 12.31s/it]
infer : od-8220                                             |  39%|###8      || 773/2000 [2:38:45<4:11:36, 12.30s/it]
infer : od-8220                                             |  39%|###8      || 774/2000 [2:38:57<4:11:30, 12.31s/it]
infer : od-8220                                             |  39%|###8      || 775/2000 [2:39:10<4:11:13, 12.30s/it]
infer : od-8220                                             |  39%|###8      || 776/2000 [2:39:22<4:10:53, 12.30s/it]
infer : od-8220                                             |  39%|###8      || 777/2000 [2:39:34<4:10:47, 12.30s/it]
infer : od-8220                                             |  39%|###8      || 778/2000 [2:39:46<4:10:37, 12.31s/it]
infer : od-8220                                             |  39%|###8      || 779/2000 [2:39:59<4:10:26, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 780/2000 [2:40:11<4:10:12, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 781/2000 [2:40:23<4:09:59, 12.30s/it]
infer : od-8220                                             |  39%|###9      || 782/2000 [2:40:36<4:09:50, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 783/2000 [2:40:48<4:09:42, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 784/2000 [2:41:00<4:09:25, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 785/2000 [2:41:13<4:09:11, 12.31s/it]
infer : od-8220                                             |  39%|###9      || 786/2000 [2:41:25<4:08:53, 12.30s/it]
infer : od-8220                                             |  39%|###9      || 787/2000 [2:41:37<4:08:42, 12.30s/it]
infer : od-8220                                             |  39%|###9      || 788/2000 [2:41:49<4:08:28, 12.30s/it]
infer : od-8220                                             |  39%|###9      || 789/2000 [2:42:02<4:08:16, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 790/2000 [2:42:14<4:08:06, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 791/2000 [2:42:26<4:07:47, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 792/2000 [2:42:39<4:07:40, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 793/2000 [2:42:51<4:07:31, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 794/2000 [2:43:03<4:07:20, 12.31s/it]
infer : od-8220                                             |  40%|###9      || 795/2000 [2:43:16<4:07:12, 12.31s/it]
infer : od-8220                                             |  40%|###9      || 796/2000 [2:43:28<4:06:55, 12.31s/it]
infer : od-8220                                             |  40%|###9      || 797/2000 [2:43:40<4:06:45, 12.31s/it]
infer : od-8220                                             |  40%|###9      || 798/2000 [2:43:53<4:06:29, 12.30s/it]
infer : od-8220                                             |  40%|###9      || 799/2000 [2:44:05<4:06:17, 12.30s/it]
infer : od-8220                                             |  40%|####      || 800/2000 [2:44:17<4:06:08, 12.31s/it]
infer : od-8220                                             |  40%|####      || 801/2000 [2:44:29<4:05:54, 12.31s/it]
infer : od-8220                                             |  40%|####      || 802/2000 [2:44:42<4:05:35, 12.30s/it]
infer : od-8220                                             |  40%|####      || 803/2000 [2:44:54<4:05:24, 12.30s/it]
infer : od-8220                                             |  40%|####      || 804/2000 [2:45:06<4:05:16, 12.30s/it]
infer : od-8220                                             |  40%|####      || 805/2000 [2:45:19<4:05:06, 12.31s/it]
infer : od-8220                                             |  40%|####      || 806/2000 [2:45:31<4:05:00, 12.31s/it]
infer : od-8220                                             |  40%|####      || 807/2000 [2:45:43<4:04:47, 12.31s/it]
infer : od-8220                                             |  40%|####      || 808/2000 [2:45:56<4:04:31, 12.31s/it]
infer : od-8220                                             |  40%|####      || 809/2000 [2:46:08<4:04:18, 12.31s/it]
infer : od-8220                                             |  40%|####      || 810/2000 [2:46:20<4:04:18, 12.32s/it]
infer : od-8220                                             |  41%|####      || 811/2000 [2:46:33<4:04:13, 12.32s/it]
infer : od-8220                                             |  41%|####      || 812/2000 [2:46:45<4:03:55, 12.32s/it]
infer : od-8220                                             |  41%|####      || 813/2000 [2:46:57<4:03:37, 12.31s/it]
infer : od-8220                                             |  41%|####      || 814/2000 [2:47:10<4:03:19, 12.31s/it]
infer : od-8220                                             |  41%|####      || 815/2000 [2:47:22<4:03:12, 12.31s/it]
infer : od-8220                                             |  41%|####      || 816/2000 [2:47:34<4:03:04, 12.32s/it]
infer : od-8220                                             |  41%|####      || 817/2000 [2:47:46<4:02:57, 12.32s/it]
infer : od-8220                                             |  41%|####      || 818/2000 [2:47:59<4:02:44, 12.32s/it]
infer : od-8220                                             |  41%|####      || 819/2000 [2:48:11<4:02:31, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 820/2000 [2:48:23<4:02:14, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 821/2000 [2:48:36<4:02:02, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 822/2000 [2:48:48<4:01:49, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 823/2000 [2:49:00<4:01:42, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 824/2000 [2:49:13<4:01:26, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 825/2000 [2:49:25<4:01:06, 12.31s/it]
infer : od-8220                                             |  41%|####1     || 826/2000 [2:49:37<4:00:58, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 827/2000 [2:49:50<4:00:51, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 828/2000 [2:50:02<4:00:35, 12.32s/it]
infer : od-8220                                             |  41%|####1     || 829/2000 [2:50:14<4:00:13, 12.31s/it]
infer : od-8220                                             |  42%|####1     || 830/2000 [2:50:27<3:59:59, 12.31s/it]
infer : od-8220                                             |  42%|####1     || 831/2000 [2:50:39<3:59:54, 12.31s/it]
infer : od-8220                                             |  42%|####1     || 832/2000 [2:50:51<3:59:42, 12.31s/it]
infer : od-8220                                             |  42%|####1     || 833/2000 [2:51:04<3:59:35, 12.32s/it]
infer : od-8220                                             |  42%|####1     || 834/2000 [2:51:16<3:59:27, 12.32s/it]
infer : od-8220                                             |  42%|####1     || 835/2000 [2:51:28<3:59:19, 12.33s/it]
infer : od-8220                                             |  42%|####1     || 836/2000 [2:51:41<3:59:10, 12.33s/it]
infer : od-8220                                             |  42%|####1     || 837/2000 [2:51:53<3:59:03, 12.33s/it]
infer : od-8220                                             |  42%|####1     || 838/2000 [2:52:05<3:58:52, 12.33s/it]
infer : od-8220                                             |  42%|####1     || 839/2000 [2:52:18<3:58:40, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 840/2000 [2:52:30<3:58:28, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 841/2000 [2:52:42<3:58:10, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 842/2000 [2:52:55<3:58:06, 12.34s/it]
infer : od-8220                                             |  42%|####2     || 843/2000 [2:53:07<3:57:58, 12.34s/it]
infer : od-8220                                             |  42%|####2     || 844/2000 [2:53:19<3:57:46, 12.34s/it]
infer : od-8220                                             |  42%|####2     || 845/2000 [2:53:32<3:57:21, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 846/2000 [2:53:44<3:57:06, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 847/2000 [2:53:56<3:56:54, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 848/2000 [2:54:09<3:56:40, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 849/2000 [2:54:21<3:56:26, 12.33s/it]
infer : od-8220                                             |  42%|####2     || 850/2000 [2:54:33<3:56:20, 12.33s/it]
infer : od-8220                                             |  43%|####2     || 851/2000 [2:54:46<3:56:00, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 852/2000 [2:54:58<3:55:45, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 853/2000 [2:55:10<3:55:31, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 854/2000 [2:55:22<3:55:15, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 855/2000 [2:55:35<3:55:00, 12.31s/it]
infer : od-8220                                             |  43%|####2     || 856/2000 [2:55:47<3:54:51, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 857/2000 [2:55:59<3:54:36, 12.32s/it]
infer : od-8220                                             |  43%|####2     || 858/2000 [2:56:12<3:54:21, 12.31s/it]
infer : od-8220                                             |  43%|####2     || 859/2000 [2:56:24<3:54:13, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 860/2000 [2:56:36<3:54:02, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 861/2000 [2:56:49<3:53:53, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 862/2000 [2:57:01<3:53:36, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 863/2000 [2:57:13<3:53:19, 12.31s/it]
infer : od-8220                                             |  43%|####3     || 864/2000 [2:57:26<3:53:05, 12.31s/it]
infer : od-8220                                             |  43%|####3     || 865/2000 [2:57:38<3:52:59, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 866/2000 [2:57:50<3:52:54, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 867/2000 [2:58:03<3:52:36, 12.32s/it]
infer : od-8220                                             |  43%|####3     || 868/2000 [2:58:15<3:52:11, 12.31s/it]
infer : od-8220                                             |  43%|####3     || 869/2000 [2:58:27<3:51:56, 12.30s/it]
infer : od-8220                                             |  44%|####3     || 870/2000 [2:58:39<3:51:50, 12.31s/it]
infer : od-8220                                             |  44%|####3     || 871/2000 [2:58:52<3:51:50, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 872/2000 [2:59:04<3:51:39, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 873/2000 [2:59:16<3:51:26, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 874/2000 [2:59:29<3:51:13, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 875/2000 [2:59:41<3:51:05, 12.33s/it]
infer : od-8220                                             |  44%|####3     || 876/2000 [2:59:53<3:50:51, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 877/2000 [3:00:06<3:50:40, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 878/2000 [3:00:18<3:50:19, 12.32s/it]
infer : od-8220                                             |  44%|####3     || 879/2000 [3:00:30<3:50:00, 12.31s/it]
infer : od-8220                                             |  44%|####4     || 880/2000 [3:00:43<3:49:53, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 881/2000 [3:00:55<3:49:54, 12.33s/it]
infer : od-8220                                             |  44%|####4     || 882/2000 [3:01:07<3:49:39, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 883/2000 [3:01:20<3:49:21, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 884/2000 [3:01:32<3:49:10, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 885/2000 [3:01:44<3:48:55, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 886/2000 [3:01:57<3:48:43, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 887/2000 [3:02:09<3:48:29, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 888/2000 [3:02:21<3:48:14, 12.32s/it]
infer : od-8220                                             |  44%|####4     || 889/2000 [3:02:34<3:48:00, 12.31s/it]
infer : od-8220                                             |  44%|####4     || 890/2000 [3:02:46<3:47:48, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 891/2000 [3:02:58<3:47:30, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 892/2000 [3:03:10<3:47:16, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 893/2000 [3:03:23<3:46:58, 12.30s/it]
infer : od-8220                                             |  45%|####4     || 894/2000 [3:03:35<3:46:53, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 895/2000 [3:03:47<3:46:42, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 896/2000 [3:04:00<3:46:25, 12.31s/it]
infer : od-8220                                             |  45%|####4     || 897/2000 [3:04:12<3:46:04, 12.30s/it]
infer : od-8220                                             |  45%|####4     || 898/2000 [3:04:24<3:45:51, 12.30s/it]
infer : od-8220                                             |  45%|####4     || 899/2000 [3:04:37<3:45:38, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 900/2000 [3:04:49<3:45:23, 12.29s/it]
infer : od-8220                                             |  45%|####5     || 901/2000 [3:05:01<3:45:16, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 902/2000 [3:05:13<3:45:02, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 903/2000 [3:05:26<3:44:50, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 904/2000 [3:05:38<3:44:51, 12.31s/it]
infer : od-8220                                             |  45%|####5     || 905/2000 [3:05:50<3:44:35, 12.31s/it]
infer : od-8220                                             |  45%|####5     || 906/2000 [3:06:03<3:44:22, 12.31s/it]
infer : od-8220                                             |  45%|####5     || 907/2000 [3:06:15<3:44:00, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 908/2000 [3:06:27<3:43:51, 12.30s/it]
infer : od-8220                                             |  45%|####5     || 909/2000 [3:06:40<3:43:38, 12.30s/it]
infer : od-8220                                             |  46%|####5     || 910/2000 [3:06:52<3:43:20, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 911/2000 [3:07:04<3:43:05, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 912/2000 [3:07:16<3:42:48, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 913/2000 [3:07:29<3:42:41, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 914/2000 [3:07:41<3:42:26, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 915/2000 [3:07:53<3:42:13, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 916/2000 [3:08:06<3:42:01, 12.29s/it]
infer : od-8220                                             |  46%|####5     || 917/2000 [3:08:18<3:41:40, 12.28s/it]
infer : od-8220                                             |  46%|####5     || 918/2000 [3:08:30<3:41:31, 12.28s/it]
infer : od-8220                                             |  46%|####5     || 919/2000 [3:08:42<3:41:10, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 920/2000 [3:08:55<3:41:02, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 921/2000 [3:09:07<3:40:48, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 922/2000 [3:09:19<3:40:39, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 923/2000 [3:09:32<3:40:27, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 924/2000 [3:09:44<3:40:17, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 925/2000 [3:09:56<3:40:07, 12.29s/it]
INFO:20250320-024919: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025019: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025119: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025219: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025319: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025419: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025519: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025619: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025719: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025819: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-025920: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030020: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030120: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030220: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030320: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030420: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030520: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030620: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030720: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030820: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-030920: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031020: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031120: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031220: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031320: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031420: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031520: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031620: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031720: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031820: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-031920: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032020: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032120: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032220: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032320: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032420: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032520: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032620: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032720: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032820: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-032920: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033020: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033120: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033220: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033320: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033420: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033520: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033620: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033720: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033820: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-033920: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034020: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034120: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034220: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034320: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034421: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034521: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034621: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034721: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034821: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-034921: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035021: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035121: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  46%|####6     || 926/2000 [3:10:08<3:39:52, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 927/2000 [3:10:21<3:39:39, 12.28s/it]
infer : od-8220                                             |  46%|####6     || 928/2000 [3:10:33<3:39:29, 12.29s/it]
infer : od-8220                                             |  46%|####6     || 929/2000 [3:10:45<3:39:19, 12.29s/it]
infer : od-8220                                             |  46%|####6     || 930/2000 [3:10:58<3:39:14, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 931/2000 [3:11:10<3:38:57, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 932/2000 [3:11:22<3:38:45, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 933/2000 [3:11:34<3:38:26, 12.28s/it]
infer : od-8220                                             |  47%|####6     || 934/2000 [3:11:47<3:38:22, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 935/2000 [3:11:59<3:38:09, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 936/2000 [3:12:11<3:38:03, 12.30s/it]
infer : od-8220                                             |  47%|####6     || 937/2000 [3:12:24<3:37:53, 12.30s/it]
infer : od-8220                                             |  47%|####6     || 938/2000 [3:12:36<3:37:44, 12.30s/it]
infer : od-8220                                             |  47%|####6     || 939/2000 [3:12:48<3:37:22, 12.29s/it]
infer : od-8220                                             |  47%|####6     || 940/2000 [3:13:00<3:36:58, 12.28s/it]
infer : od-8220                                             |  47%|####7     || 941/2000 [3:13:13<3:36:46, 12.28s/it]
infer : od-8220                                             |  47%|####7     || 942/2000 [3:13:25<3:36:36, 12.28s/it]
infer : od-8220                                             |  47%|####7     || 943/2000 [3:13:37<3:36:25, 12.29s/it]
infer : od-8220                                             |  47%|####7     || 944/2000 [3:13:50<3:36:08, 12.28s/it]
infer : od-8220                                             |  47%|####7     || 945/2000 [3:14:02<3:35:49, 12.27s/it]
infer : od-8220                                             |  47%|####7     || 946/2000 [3:14:14<3:35:46, 12.28s/it]
infer : od-8220                                             |  47%|####7     || 947/2000 [3:14:26<3:35:44, 12.29s/it]
infer : od-8220                                             |  47%|####7     || 948/2000 [3:14:39<3:35:32, 12.29s/it]
infer : od-8220                                             |  47%|####7     || 949/2000 [3:14:51<3:35:20, 12.29s/it]
infer : od-8220                                             |  48%|####7     || 950/2000 [3:15:03<3:34:59, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 951/2000 [3:15:16<3:34:46, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 952/2000 [3:15:28<3:34:33, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 953/2000 [3:15:40<3:34:16, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 954/2000 [3:15:52<3:34:04, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 955/2000 [3:16:05<3:33:56, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 956/2000 [3:16:17<3:33:34, 12.27s/it]
infer : od-8220                                             |  48%|####7     || 957/2000 [3:16:29<3:33:25, 12.28s/it]
infer : od-8220                                             |  48%|####7     || 958/2000 [3:16:42<3:33:30, 12.29s/it]
infer : od-8220                                             |  48%|####7     || 959/2000 [3:16:54<3:33:17, 12.29s/it]
infer : od-8220                                             |  48%|####8     || 960/2000 [3:17:06<3:32:55, 12.28s/it]
infer : od-8220                                             |  48%|####8     || 961/2000 [3:17:18<3:32:38, 12.28s/it]
infer : od-8220                                             |  48%|####8     || 962/2000 [3:17:31<3:32:31, 12.29s/it]
infer : od-8220                                             |  48%|####8     || 963/2000 [3:17:43<3:32:29, 12.30s/it]
infer : od-8220                                             |  48%|####8     || 964/2000 [3:17:55<3:32:19, 12.30s/it]
infer : od-8220                                             |  48%|####8     || 965/2000 [3:18:08<3:32:13, 12.30s/it]
infer : od-8220                                             |  48%|####8     || 966/2000 [3:18:20<3:31:54, 12.30s/it]
infer : od-8220                                             |  48%|####8     || 967/2000 [3:18:32<3:31:40, 12.29s/it]
infer : od-8220                                             |  48%|####8     || 968/2000 [3:18:45<3:31:24, 12.29s/it]
infer : od-8220                                             |  48%|####8     || 969/2000 [3:18:57<3:31:02, 12.28s/it]
infer : od-8220                                             |  48%|####8     || 970/2000 [3:19:09<3:30:49, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 971/2000 [3:19:21<3:30:31, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 972/2000 [3:19:34<3:30:19, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 973/2000 [3:19:46<3:30:07, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 974/2000 [3:19:58<3:29:59, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 975/2000 [3:20:10<3:29:49, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 976/2000 [3:20:23<3:29:36, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 977/2000 [3:20:35<3:29:19, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 978/2000 [3:20:47<3:29:09, 12.28s/it]
infer : od-8220                                             |  49%|####8     || 979/2000 [3:21:00<3:29:01, 12.28s/it]
infer : od-8220                                             |  49%|####9     || 980/2000 [3:21:12<3:28:50, 12.28s/it]
infer : od-8220                                             |  49%|####9     || 981/2000 [3:21:24<3:28:35, 12.28s/it]
infer : od-8220                                             |  49%|####9     || 982/2000 [3:21:36<3:28:29, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 983/2000 [3:21:49<3:28:18, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 984/2000 [3:22:01<3:28:08, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 985/2000 [3:22:13<3:27:50, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 986/2000 [3:22:26<3:27:36, 12.28s/it]
infer : od-8220                                             |  49%|####9     || 987/2000 [3:22:38<3:27:28, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 988/2000 [3:22:50<3:27:21, 12.29s/it]
infer : od-8220                                             |  49%|####9     || 989/2000 [3:23:02<3:27:02, 12.29s/it]
infer : od-8220                                             |  50%|####9     || 990/2000 [3:23:15<3:26:51, 12.29s/it]
infer : od-8220                                             |  50%|####9     || 991/2000 [3:23:27<3:26:53, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 992/2000 [3:23:39<3:26:36, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 993/2000 [3:23:52<3:26:22, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 994/2000 [3:24:04<3:26:08, 12.29s/it]
infer : od-8220                                             |  50%|####9     || 995/2000 [3:24:16<3:26:01, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 996/2000 [3:24:29<3:25:47, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 997/2000 [3:24:41<3:25:33, 12.30s/it]
infer : od-8220                                             |  50%|####9     || 998/2000 [3:24:53<3:25:14, 12.29s/it]
infer : od-8220                                             |  50%|####9     || 999/2000 [3:25:05<3:24:54, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1000/2000 [3:25:18<3:24:37, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1001/2000 [3:25:30<3:24:27, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1002/2000 [3:25:42<3:24:11, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1003/2000 [3:25:54<3:23:59, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1004/2000 [3:26:07<3:23:43, 12.27s/it]
infer : od-8220                                             |  50%|#####     || 1005/2000 [3:26:19<3:23:27, 12.27s/it]
infer : od-8220                                             |  50%|#####     || 1006/2000 [3:26:31<3:23:27, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1007/2000 [3:26:44<3:23:17, 12.28s/it]
infer : od-8220                                             |  50%|#####     || 1008/2000 [3:26:56<3:23:08, 12.29s/it]
infer : od-8220                                             |  50%|#####     || 1009/2000 [3:27:08<3:22:55, 12.29s/it]
infer : od-8220                                             |  50%|#####     || 1010/2000 [3:27:20<3:22:41, 12.28s/it]
infer : od-8220                                             |  51%|#####     || 1011/2000 [3:27:33<3:22:34, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1012/2000 [3:27:45<3:22:24, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1013/2000 [3:27:57<3:22:13, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1014/2000 [3:28:10<3:21:58, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1015/2000 [3:28:22<3:21:41, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1016/2000 [3:28:34<3:21:27, 12.28s/it]
infer : od-8220                                             |  51%|#####     || 1017/2000 [3:28:46<3:21:13, 12.28s/it]
infer : od-8220                                             |  51%|#####     || 1018/2000 [3:28:59<3:21:10, 12.29s/it]
infer : od-8220                                             |  51%|#####     || 1019/2000 [3:29:11<3:20:58, 12.29s/it]
infer : od-8220                                             |  51%|#####1    || 1020/2000 [3:29:23<3:20:44, 12.29s/it]
infer : od-8220                                             |  51%|#####1    || 1021/2000 [3:29:36<3:20:27, 12.28s/it]
infer : od-8220                                             |  51%|#####1    || 1022/2000 [3:29:48<3:20:12, 12.28s/it]
infer : od-8220                                             |  51%|#####1    || 1023/2000 [3:30:00<3:20:02, 12.29s/it]
infer : od-8220                                             |  51%|#####1    || 1024/2000 [3:30:13<3:19:57, 12.29s/it]
infer : od-8220                                             |  51%|#####1    || 1025/2000 [3:30:25<3:19:54, 12.30s/it]
infer : od-8220                                             |  51%|#####1    || 1026/2000 [3:30:37<3:19:42, 12.30s/it]
infer : od-8220                                             |  51%|#####1    || 1027/2000 [3:30:49<3:19:33, 12.31s/it]
infer : od-8220                                             |  51%|#####1    || 1028/2000 [3:31:02<3:19:38, 12.32s/it]
infer : od-8220                                             |  51%|#####1    || 1029/2000 [3:31:14<3:19:21, 12.32s/it]
infer : od-8220                                             |  52%|#####1    || 1030/2000 [3:31:26<3:19:07, 12.32s/it]
infer : od-8220                                             |  52%|#####1    || 1031/2000 [3:31:39<3:18:46, 12.31s/it]
infer : od-8220                                             |  52%|#####1    || 1032/2000 [3:31:51<3:18:46, 12.32s/it]
infer : od-8220                                             |  52%|#####1    || 1033/2000 [3:32:03<3:18:22, 12.31s/it]
infer : od-8220                                             |  52%|#####1    || 1034/2000 [3:32:16<3:18:07, 12.31s/it]
infer : od-8220                                             |  52%|#####1    || 1035/2000 [3:32:28<3:18:00, 12.31s/it]
infer : od-8220                                             |  52%|#####1    || 1036/2000 [3:32:40<3:17:38, 12.30s/it]
infer : od-8220                                             |  52%|#####1    || 1037/2000 [3:32:53<3:17:20, 12.30s/it]
infer : od-8220                                             |  52%|#####1    || 1038/2000 [3:33:05<3:16:58, 12.29s/it]
infer : od-8220                                             |  52%|#####1    || 1039/2000 [3:33:17<3:16:44, 12.28s/it]
infer : od-8220                                             |  52%|#####2    || 1040/2000 [3:33:29<3:16:46, 12.30s/it]
infer : od-8220                                             |  52%|#####2    || 1041/2000 [3:33:42<3:16:44, 12.31s/it]
infer : od-8220                                             |  52%|#####2    || 1042/2000 [3:33:54<3:16:35, 12.31s/it]
infer : od-8220                                             |  52%|#####2    || 1043/2000 [3:34:06<3:16:15, 12.30s/it]
infer : od-8220                                             |  52%|#####2    || 1044/2000 [3:34:19<3:16:07, 12.31s/it]
infer : od-8220                                             |  52%|#####2    || 1045/2000 [3:34:31<3:16:02, 12.32s/it]
infer : od-8220                                             |  52%|#####2    || 1046/2000 [3:34:43<3:15:50, 12.32s/it]
infer : od-8220                                             |  52%|#####2    || 1047/2000 [3:34:56<3:15:38, 12.32s/it]
infer : od-8220                                             |  52%|#####2    || 1048/2000 [3:35:08<3:15:22, 12.31s/it]
infer : od-8220                                             |  52%|#####2    || 1049/2000 [3:35:20<3:15:11, 12.31s/it]
infer : od-8220                                             |  52%|#####2    || 1050/2000 [3:35:33<3:15:02, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1051/2000 [3:35:45<3:14:53, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1052/2000 [3:35:57<3:14:41, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1053/2000 [3:36:10<3:14:24, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1054/2000 [3:36:22<3:14:07, 12.31s/it]
infer : od-8220                                             |  53%|#####2    || 1055/2000 [3:36:34<3:14:01, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1056/2000 [3:36:47<3:13:49, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1057/2000 [3:36:59<3:13:36, 12.32s/it]
infer : od-8220                                             |  53%|#####2    || 1058/2000 [3:37:11<3:13:19, 12.31s/it]
infer : od-8220                                             |  53%|#####2    || 1059/2000 [3:37:23<3:13:00, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1060/2000 [3:37:36<3:12:54, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1061/2000 [3:37:48<3:12:42, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1062/2000 [3:38:00<3:12:25, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1063/2000 [3:38:13<3:12:14, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1064/2000 [3:38:25<3:11:59, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1065/2000 [3:38:37<3:11:41, 12.30s/it]
infer : od-8220                                             |  53%|#####3    || 1066/2000 [3:38:50<3:11:35, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1067/2000 [3:39:02<3:11:26, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1068/2000 [3:39:14<3:11:17, 12.31s/it]
infer : od-8220                                             |  53%|#####3    || 1069/2000 [3:39:27<3:11:00, 12.31s/it]
infer : od-8220                                             |  54%|#####3    || 1070/2000 [3:39:39<3:10:53, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1071/2000 [3:39:51<3:10:41, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1072/2000 [3:40:04<3:10:29, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1073/2000 [3:40:16<3:10:15, 12.31s/it]
infer : od-8220                                             |  54%|#####3    || 1074/2000 [3:40:28<3:10:04, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1075/2000 [3:40:40<3:09:59, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1076/2000 [3:40:53<3:09:47, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1077/2000 [3:41:05<3:09:33, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1078/2000 [3:41:17<3:09:20, 12.32s/it]
infer : od-8220                                             |  54%|#####3    || 1079/2000 [3:41:30<3:09:09, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1080/2000 [3:41:42<3:08:54, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1081/2000 [3:41:54<3:08:39, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1082/2000 [3:42:07<3:08:28, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1083/2000 [3:42:19<3:08:16, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1084/2000 [3:42:31<3:08:08, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1085/2000 [3:42:44<3:07:51, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1086/2000 [3:42:56<3:07:32, 12.31s/it]
infer : od-8220                                             |  54%|#####4    || 1087/2000 [3:43:08<3:07:24, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1088/2000 [3:43:21<3:07:12, 12.32s/it]
infer : od-8220                                             |  54%|#####4    || 1089/2000 [3:43:33<3:06:57, 12.31s/it]
infer : od-8220                                             |  55%|#####4    || 1090/2000 [3:43:45<3:06:50, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1091/2000 [3:43:58<3:06:34, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1092/2000 [3:44:10<3:06:20, 12.31s/it]
infer : od-8220                                             |  55%|#####4    || 1093/2000 [3:44:22<3:06:10, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1094/2000 [3:44:35<3:06:03, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1095/2000 [3:44:47<3:05:49, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1096/2000 [3:44:59<3:05:36, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1097/2000 [3:45:11<3:05:19, 12.31s/it]
infer : od-8220                                             |  55%|#####4    || 1098/2000 [3:45:24<3:05:08, 12.32s/it]
infer : od-8220                                             |  55%|#####4    || 1099/2000 [3:45:36<3:05:01, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1100/2000 [3:45:48<3:04:48, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1101/2000 [3:46:01<3:04:36, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1102/2000 [3:46:13<3:04:21, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1103/2000 [3:46:25<3:04:10, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1104/2000 [3:46:38<3:03:59, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1105/2000 [3:46:50<3:03:45, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1106/2000 [3:47:02<3:03:32, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1107/2000 [3:47:15<3:03:21, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1108/2000 [3:47:27<3:03:09, 12.32s/it]
infer : od-8220                                             |  55%|#####5    || 1109/2000 [3:47:39<3:03:00, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1110/2000 [3:47:52<3:02:51, 12.33s/it]
infer : od-8220                                             |  56%|#####5    || 1111/2000 [3:48:04<3:02:43, 12.33s/it]
infer : od-8220                                             |  56%|#####5    || 1112/2000 [3:48:16<3:02:28, 12.33s/it]
infer : od-8220                                             |  56%|#####5    || 1113/2000 [3:48:29<3:02:11, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1114/2000 [3:48:41<3:01:59, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1115/2000 [3:48:53<3:01:49, 12.33s/it]
infer : od-8220                                             |  56%|#####5    || 1116/2000 [3:49:06<3:01:31, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1117/2000 [3:49:18<3:01:14, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1118/2000 [3:49:30<3:01:02, 12.32s/it]
infer : od-8220                                             |  56%|#####5    || 1119/2000 [3:49:43<3:00:52, 12.32s/it]
infer : od-8220                                             |  56%|#####6    || 1120/2000 [3:49:55<3:00:35, 12.31s/it]
infer : od-8220                                             |  56%|#####6    || 1121/2000 [3:50:07<3:00:18, 12.31s/it]
infer : od-8220                                             |  56%|#####6    || 1122/2000 [3:50:19<3:00:04, 12.31s/it]
infer : od-8220                                             |  56%|#####6    || 1123/2000 [3:50:32<2:59:55, 12.31s/it]
infer : od-8220                                             |  56%|#####6    || 1124/2000 [3:50:44<2:59:45, 12.31s/it]
infer : od-8220                                             |  56%|#####6    || 1125/2000 [3:50:56<2:59:41, 12.32s/it]
infer : od-8220                                             |  56%|#####6    || 1126/2000 [3:51:09<2:59:31, 12.32s/it]
infer : od-8220                                             |  56%|#####6    || 1127/2000 [3:51:21<2:59:19, 12.33s/it]
infer : od-8220                                             |  56%|#####6    || 1128/2000 [3:51:33<2:59:03, 12.32s/it]
infer : od-8220                                             |  56%|#####6    || 1129/2000 [3:51:46<2:58:52, 12.32s/it]
infer : od-8220                                             |  56%|#####6    || 1130/2000 [3:51:58<2:58:42, 12.33s/it]
infer : od-8220                                             |  57%|#####6    || 1131/2000 [3:52:10<2:58:30, 12.33s/it]
infer : od-8220                                             |  57%|#####6    || 1132/2000 [3:52:23<2:58:17, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1133/2000 [3:52:35<2:58:03, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1134/2000 [3:52:47<2:57:46, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1135/2000 [3:53:00<2:57:36, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1136/2000 [3:53:12<2:57:24, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1137/2000 [3:53:24<2:57:10, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1138/2000 [3:53:37<2:56:57, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1139/2000 [3:53:49<2:56:47, 12.32s/it]
infer : od-8220                                             |  57%|#####6    || 1140/2000 [3:54:01<2:56:38, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1141/2000 [3:54:14<2:56:21, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1142/2000 [3:54:26<2:56:12, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1143/2000 [3:54:38<2:56:03, 12.33s/it]
infer : od-8220                                             |  57%|#####7    || 1144/2000 [3:54:51<2:55:47, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1145/2000 [3:55:03<2:55:27, 12.31s/it]
infer : od-8220                                             |  57%|#####7    || 1146/2000 [3:55:15<2:55:15, 12.31s/it]
infer : od-8220                                             |  57%|#####7    || 1147/2000 [3:55:27<2:55:09, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1148/2000 [3:55:40<2:54:53, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1149/2000 [3:55:52<2:54:40, 12.32s/it]
infer : od-8220                                             |  57%|#####7    || 1150/2000 [3:56:04<2:54:25, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1151/2000 [3:56:17<2:54:15, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1152/2000 [3:56:29<2:54:03, 12.32s/it]
infer : od-8220                                             |  58%|#####7    || 1153/2000 [3:56:41<2:53:50, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1154/2000 [3:56:54<2:53:38, 12.32s/it]
infer : od-8220                                             |  58%|#####7    || 1155/2000 [3:57:06<2:53:22, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1156/2000 [3:57:18<2:53:09, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1157/2000 [3:57:31<2:52:59, 12.31s/it]
infer : od-8220                                             |  58%|#####7    || 1158/2000 [3:57:43<2:52:49, 12.32s/it]
infer : od-8220                                             |  58%|#####7    || 1159/2000 [3:57:55<2:52:34, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1160/2000 [3:58:08<2:52:20, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1161/2000 [3:58:20<2:52:06, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1162/2000 [3:58:32<2:51:59, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1163/2000 [3:58:45<2:51:57, 12.33s/it]
infer : od-8220                                             |  58%|#####8    || 1164/2000 [3:58:57<2:51:43, 12.32s/it]
infer : od-8220                                             |  58%|#####8    || 1165/2000 [3:59:09<2:51:24, 12.32s/it]
infer : od-8220                                             |  58%|#####8    || 1166/2000 [3:59:21<2:51:10, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1167/2000 [3:59:34<2:50:57, 12.31s/it]
infer : od-8220                                             |  58%|#####8    || 1168/2000 [3:59:46<2:50:51, 12.32s/it]
infer : od-8220                                             |  58%|#####8    || 1169/2000 [3:59:58<2:50:37, 12.32s/it]
infer : od-8220                                             |  58%|#####8    || 1170/2000 [4:00:11<2:50:24, 12.32s/it]
infer : od-8220                                             |  59%|#####8    || 1171/2000 [4:00:23<2:50:09, 12.32s/it]
infer : od-8220                                             |  59%|#####8    || 1172/2000 [4:00:35<2:49:56, 12.31s/it]
infer : od-8220                                             |  59%|#####8    || 1173/2000 [4:00:48<2:49:49, 12.32s/it]
infer : od-8220                                             |  59%|#####8    || 1174/2000 [4:01:00<2:49:40, 12.33s/it]
infer : od-8220                                             |  59%|#####8    || 1175/2000 [4:01:12<2:49:23, 12.32s/it]
infer : od-8220                                             |  59%|#####8    || 1176/2000 [4:01:25<2:49:10, 12.32s/it]
infer : od-8220                                             |  59%|#####8    || 1177/2000 [4:01:37<2:48:51, 12.31s/it]
infer : od-8220                                             |  59%|#####8    || 1178/2000 [4:01:49<2:48:39, 12.31s/it]
infer : od-8220                                             |  59%|#####8    || 1179/2000 [4:02:02<2:48:28, 12.31s/it]
infer : od-8220                                             |  59%|#####8    || 1180/2000 [4:02:14<2:48:18, 12.32s/it]
infer : od-8220                                             |  59%|#####9    || 1181/2000 [4:02:26<2:48:06, 12.32s/it]
infer : od-8220                                             |  59%|#####9    || 1182/2000 [4:02:39<2:47:53, 12.31s/it]
infer : od-8220                                             |  59%|#####9    || 1183/2000 [4:02:51<2:47:41, 12.32s/it]
infer : od-8220                                             |  59%|#####9    || 1184/2000 [4:03:03<2:47:27, 12.31s/it]
infer : od-8220                                             |  59%|#####9    || 1185/2000 [4:03:15<2:47:12, 12.31s/it]
infer : od-8220                                             |  59%|#####9    || 1186/2000 [4:03:28<2:46:58, 12.31s/it]
infer : od-8220                                             |  59%|#####9    || 1187/2000 [4:03:40<2:46:49, 12.31s/it]
infer : od-8220                                             |  59%|#####9    || 1188/2000 [4:03:52<2:46:46, 12.32s/it]
infer : od-8220                                             |  59%|#####9    || 1189/2000 [4:04:05<2:46:36, 12.33s/it]
infer : od-8220                                             |  60%|#####9    || 1190/2000 [4:04:17<2:46:22, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1191/2000 [4:04:29<2:46:11, 12.33s/it]
infer : od-8220                                             |  60%|#####9    || 1192/2000 [4:04:42<2:45:57, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1193/2000 [4:04:54<2:45:44, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1194/2000 [4:05:06<2:45:34, 12.33s/it]
infer : od-8220                                             |  60%|#####9    || 1195/2000 [4:05:19<2:45:21, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1196/2000 [4:05:31<2:45:08, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1197/2000 [4:05:43<2:44:55, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1198/2000 [4:05:56<2:44:39, 12.32s/it]
infer : od-8220                                             |  60%|#####9    || 1199/2000 [4:06:08<2:44:27, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1200/2000 [4:06:20<2:44:16, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1201/2000 [4:06:33<2:44:02, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1202/2000 [4:06:45<2:43:52, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1203/2000 [4:06:57<2:43:37, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1204/2000 [4:07:10<2:43:24, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1205/2000 [4:07:22<2:43:10, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1206/2000 [4:07:34<2:43:04, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1207/2000 [4:07:47<2:42:48, 12.32s/it]
infer : od-8220                                             |  60%|######    || 1208/2000 [4:07:59<2:42:31, 12.31s/it]
infer : od-8220                                             |  60%|######    || 1209/2000 [4:08:11<2:42:11, 12.30s/it]
infer : od-8220                                             |  60%|######    || 1210/2000 [4:08:23<2:42:01, 12.31s/it]
infer : od-8220                                             |  61%|######    || 1211/2000 [4:08:36<2:41:51, 12.31s/it]
infer : od-8220                                             |  61%|######    || 1212/2000 [4:08:48<2:41:38, 12.31s/it]
infer : od-8220                                             |  61%|######    || 1213/2000 [4:09:00<2:41:21, 12.30s/it]
infer : od-8220                                             |  61%|######    || 1214/2000 [4:09:13<2:41:05, 12.30s/it]
infer : od-8220                                             |  61%|######    || 1215/2000 [4:09:25<2:40:57, 12.30s/it]
infer : od-8220                                             |  61%|######    || 1216/2000 [4:09:37<2:40:44, 12.30s/it]
infer : od-8220                                             |  61%|######    || 1217/2000 [4:09:50<2:40:35, 12.31s/it]
infer : od-8220                                             |  61%|######    || 1218/2000 [4:10:02<2:40:29, 12.31s/it]
infer : od-8220                                             |  61%|######    || 1219/2000 [4:10:14<2:40:19, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1220/2000 [4:10:27<2:40:06, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1221/2000 [4:10:39<2:39:59, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1222/2000 [4:10:51<2:39:46, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1223/2000 [4:11:03<2:39:34, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1224/2000 [4:11:16<2:39:19, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1225/2000 [4:11:28<2:39:04, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1226/2000 [4:11:40<2:38:56, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1227/2000 [4:11:53<2:38:43, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1228/2000 [4:12:05<2:38:30, 12.32s/it]
infer : od-8220                                             |  61%|######1   || 1229/2000 [4:12:17<2:38:18, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1230/2000 [4:12:30<2:38:03, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1231/2000 [4:12:42<2:37:51, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1232/2000 [4:12:54<2:37:38, 12.32s/it]
INFO:20250320-035221: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035321: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035421: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035521: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035621: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035721: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035821: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-035921: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040021: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040121: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040221: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040321: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040421: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040521: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040621: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040721: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040821: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-040921: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041021: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041121: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041221: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041321: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041421: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041521: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041621: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041721: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041821: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-041921: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042021: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042121: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042221: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042321: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042421: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042521: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042621: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042722: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042822: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-042922: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043022: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043122: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043222: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043322: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043422: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043522: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043622: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043722: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043822: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-043922: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044022: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044122: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044222: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044322: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044422: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044522: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044622: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044722: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044822: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-044922: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045022: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045122: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045222: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045322: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045422: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  62%|######1   || 1233/2000 [4:13:07<2:37:26, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1234/2000 [4:13:19<2:37:11, 12.31s/it]
infer : od-8220                                             |  62%|######1   || 1235/2000 [4:13:31<2:36:59, 12.31s/it]
infer : od-8220                                             |  62%|######1   || 1236/2000 [4:13:44<2:36:52, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1237/2000 [4:13:56<2:36:40, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1238/2000 [4:14:08<2:36:30, 12.32s/it]
infer : od-8220                                             |  62%|######1   || 1239/2000 [4:14:21<2:36:13, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1240/2000 [4:14:33<2:35:59, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1241/2000 [4:14:45<2:35:51, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1242/2000 [4:14:58<2:35:37, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1243/2000 [4:15:10<2:35:20, 12.31s/it]
infer : od-8220                                             |  62%|######2   || 1244/2000 [4:15:22<2:35:10, 12.31s/it]
infer : od-8220                                             |  62%|######2   || 1245/2000 [4:15:34<2:34:56, 12.31s/it]
infer : od-8220                                             |  62%|######2   || 1246/2000 [4:15:47<2:34:47, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1247/2000 [4:15:59<2:34:40, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1248/2000 [4:16:11<2:34:25, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1249/2000 [4:16:24<2:34:14, 12.32s/it]
infer : od-8220                                             |  62%|######2   || 1250/2000 [4:16:36<2:33:59, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1251/2000 [4:16:48<2:33:42, 12.31s/it]
infer : od-8220                                             |  63%|######2   || 1252/2000 [4:17:01<2:33:31, 12.31s/it]
infer : od-8220                                             |  63%|######2   || 1253/2000 [4:17:13<2:33:22, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1254/2000 [4:17:25<2:33:14, 12.33s/it]
infer : od-8220                                             |  63%|######2   || 1255/2000 [4:17:38<2:33:02, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1256/2000 [4:17:50<2:32:45, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1257/2000 [4:18:02<2:32:32, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1258/2000 [4:18:15<2:32:20, 12.32s/it]
infer : od-8220                                             |  63%|######2   || 1259/2000 [4:18:27<2:32:04, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1260/2000 [4:18:39<2:31:51, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1261/2000 [4:18:52<2:31:39, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1262/2000 [4:19:04<2:31:25, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1263/2000 [4:19:16<2:31:15, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1264/2000 [4:19:28<2:31:02, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1265/2000 [4:19:41<2:30:50, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1266/2000 [4:19:53<2:30:36, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1267/2000 [4:20:05<2:30:24, 12.31s/it]
infer : od-8220                                             |  63%|######3   || 1268/2000 [4:20:18<2:30:16, 12.32s/it]
infer : od-8220                                             |  63%|######3   || 1269/2000 [4:20:30<2:30:06, 12.32s/it]
infer : od-8220                                             |  64%|######3   || 1270/2000 [4:20:42<2:29:52, 12.32s/it]
infer : od-8220                                             |  64%|######3   || 1271/2000 [4:20:55<2:29:35, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1272/2000 [4:21:07<2:29:21, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1273/2000 [4:21:19<2:29:12, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1274/2000 [4:21:32<2:29:06, 12.32s/it]
infer : od-8220                                             |  64%|######3   || 1275/2000 [4:21:44<2:28:50, 12.32s/it]
infer : od-8220                                             |  64%|######3   || 1276/2000 [4:21:56<2:28:35, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1277/2000 [4:22:09<2:28:17, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1278/2000 [4:22:21<2:28:05, 12.31s/it]
infer : od-8220                                             |  64%|######3   || 1279/2000 [4:22:33<2:27:58, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1280/2000 [4:22:46<2:27:48, 12.32s/it]
infer : od-8220                                             |  64%|######4   || 1281/2000 [4:22:58<2:27:30, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1282/2000 [4:23:10<2:27:11, 12.30s/it]
infer : od-8220                                             |  64%|######4   || 1283/2000 [4:23:22<2:27:05, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1284/2000 [4:23:35<2:26:56, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1285/2000 [4:23:47<2:26:43, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1286/2000 [4:23:59<2:26:32, 12.32s/it]
infer : od-8220                                             |  64%|######4   || 1287/2000 [4:24:12<2:26:19, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1288/2000 [4:24:24<2:26:03, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1289/2000 [4:24:36<2:25:50, 12.31s/it]
infer : od-8220                                             |  64%|######4   || 1290/2000 [4:24:49<2:25:43, 12.32s/it]
infer : od-8220                                             |  65%|######4   || 1291/2000 [4:25:01<2:25:32, 12.32s/it]
infer : od-8220                                             |  65%|######4   || 1292/2000 [4:25:13<2:25:19, 12.32s/it]
infer : od-8220                                             |  65%|######4   || 1293/2000 [4:25:26<2:25:01, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1294/2000 [4:25:38<2:24:49, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1295/2000 [4:25:50<2:24:37, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1296/2000 [4:26:02<2:24:26, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1297/2000 [4:26:15<2:24:10, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1298/2000 [4:26:27<2:23:58, 12.31s/it]
infer : od-8220                                             |  65%|######4   || 1299/2000 [4:26:39<2:23:49, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1300/2000 [4:26:52<2:23:38, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1301/2000 [4:27:04<2:23:25, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1302/2000 [4:27:16<2:23:14, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1303/2000 [4:27:29<2:23:00, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1304/2000 [4:27:41<2:22:50, 12.31s/it]
infer : od-8220                                             |  65%|######5   || 1305/2000 [4:27:53<2:22:43, 12.32s/it]
infer : od-8220                                             |  65%|######5   || 1306/2000 [4:28:06<2:22:31, 12.32s/it]
infer : od-8220                                             |  65%|######5   || 1307/2000 [4:28:18<2:22:16, 12.32s/it]
infer : od-8220                                             |  65%|######5   || 1308/2000 [4:28:30<2:22:03, 12.32s/it]
infer : od-8220                                             |  65%|######5   || 1309/2000 [4:28:43<2:21:46, 12.31s/it]
infer : od-8220                                             |  66%|######5   || 1310/2000 [4:28:55<2:21:35, 12.31s/it]
infer : od-8220                                             |  66%|######5   || 1311/2000 [4:29:07<2:21:26, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1312/2000 [4:29:20<2:21:14, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1313/2000 [4:29:32<2:21:01, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1314/2000 [4:29:44<2:20:47, 12.31s/it]
infer : od-8220                                             |  66%|######5   || 1315/2000 [4:29:56<2:20:34, 12.31s/it]
infer : od-8220                                             |  66%|######5   || 1316/2000 [4:30:09<2:20:27, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1317/2000 [4:30:21<2:20:14, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1318/2000 [4:30:33<2:20:00, 12.32s/it]
infer : od-8220                                             |  66%|######5   || 1319/2000 [4:30:46<2:19:48, 12.32s/it]
infer : od-8220                                             |  66%|######6   || 1320/2000 [4:30:58<2:19:43, 12.33s/it]
infer : od-8220                                             |  66%|######6   || 1321/2000 [4:31:10<2:19:24, 12.32s/it]
infer : od-8220                                             |  66%|######6   || 1322/2000 [4:31:23<2:19:09, 12.32s/it]
infer : od-8220                                             |  66%|######6   || 1323/2000 [4:31:35<2:18:55, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1324/2000 [4:31:47<2:18:44, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1325/2000 [4:32:00<2:18:30, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1326/2000 [4:32:12<2:18:14, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1327/2000 [4:32:24<2:18:06, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1328/2000 [4:32:37<2:17:54, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1329/2000 [4:32:49<2:17:40, 12.31s/it]
infer : od-8220                                             |  66%|######6   || 1330/2000 [4:33:01<2:17:25, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1331/2000 [4:33:13<2:17:10, 12.30s/it]
infer : od-8220                                             |  67%|######6   || 1332/2000 [4:33:26<2:17:00, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1333/2000 [4:33:38<2:16:51, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1334/2000 [4:33:50<2:16:45, 12.32s/it]
infer : od-8220                                             |  67%|######6   || 1335/2000 [4:34:03<2:16:32, 12.32s/it]
infer : od-8220                                             |  67%|######6   || 1336/2000 [4:34:15<2:16:16, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1337/2000 [4:34:27<2:16:01, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1338/2000 [4:34:40<2:15:50, 12.31s/it]
infer : od-8220                                             |  67%|######6   || 1339/2000 [4:34:52<2:15:36, 12.31s/it]
infer : od-8220                                             |  67%|######7   || 1340/2000 [4:35:04<2:15:26, 12.31s/it]
infer : od-8220                                             |  67%|######7   || 1341/2000 [4:35:17<2:15:15, 12.32s/it]
infer : od-8220                                             |  67%|######7   || 1342/2000 [4:35:29<2:15:02, 12.31s/it]
infer : od-8220                                             |  67%|######7   || 1343/2000 [4:35:41<2:14:49, 12.31s/it]
infer : od-8220                                             |  67%|######7   || 1344/2000 [4:35:54<2:14:37, 12.31s/it]
infer : od-8220                                             |  67%|######7   || 1345/2000 [4:36:06<2:14:27, 12.32s/it]
infer : od-8220                                             |  67%|######7   || 1346/2000 [4:36:18<2:14:16, 12.32s/it]
infer : od-8220                                             |  67%|######7   || 1347/2000 [4:36:31<2:14:02, 12.32s/it]
infer : od-8220                                             |  67%|######7   || 1348/2000 [4:36:43<2:13:51, 12.32s/it]
infer : od-8220                                             |  67%|######7   || 1349/2000 [4:36:55<2:13:40, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1350/2000 [4:37:07<2:13:27, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1351/2000 [4:37:20<2:13:18, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1352/2000 [4:37:32<2:13:05, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1353/2000 [4:37:44<2:12:53, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1354/2000 [4:37:57<2:12:37, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1355/2000 [4:38:09<2:12:18, 12.31s/it]
infer : od-8220                                             |  68%|######7   || 1356/2000 [4:38:21<2:12:05, 12.31s/it]
infer : od-8220                                             |  68%|######7   || 1357/2000 [4:38:34<2:12:00, 12.32s/it]
infer : od-8220                                             |  68%|######7   || 1358/2000 [4:38:46<2:11:43, 12.31s/it]
infer : od-8220                                             |  68%|######7   || 1359/2000 [4:38:58<2:11:29, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1360/2000 [4:39:11<2:11:15, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1361/2000 [4:39:23<2:11:03, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1362/2000 [4:39:35<2:10:55, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1363/2000 [4:39:48<2:10:47, 12.32s/it]
infer : od-8220                                             |  68%|######8   || 1364/2000 [4:40:00<2:10:34, 12.32s/it]
infer : od-8220                                             |  68%|######8   || 1365/2000 [4:40:12<2:10:20, 12.32s/it]
infer : od-8220                                             |  68%|######8   || 1366/2000 [4:40:24<2:10:06, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1367/2000 [4:40:37<2:09:55, 12.31s/it]
infer : od-8220                                             |  68%|######8   || 1368/2000 [4:40:49<2:09:44, 12.32s/it]
infer : od-8220                                             |  68%|######8   || 1369/2000 [4:41:01<2:09:31, 12.32s/it]
infer : od-8220                                             |  68%|######8   || 1370/2000 [4:41:14<2:09:18, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1371/2000 [4:41:26<2:09:01, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1372/2000 [4:41:38<2:08:48, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1373/2000 [4:41:51<2:08:40, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1374/2000 [4:42:03<2:08:27, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1375/2000 [4:42:15<2:08:16, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1376/2000 [4:42:28<2:08:01, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1377/2000 [4:42:40<2:07:47, 12.31s/it]
infer : od-8220                                             |  69%|######8   || 1378/2000 [4:42:52<2:07:32, 12.30s/it]
infer : od-8220                                             |  69%|######8   || 1379/2000 [4:43:05<2:07:19, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1380/2000 [4:43:17<2:07:09, 12.31s/it]
infer : od-8220                                             |  69%|######9   || 1381/2000 [4:43:29<2:06:57, 12.31s/it]
infer : od-8220                                             |  69%|######9   || 1382/2000 [4:43:41<2:06:41, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1383/2000 [4:43:54<2:06:28, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1384/2000 [4:44:06<2:06:14, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1385/2000 [4:44:18<2:06:04, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1386/2000 [4:44:31<2:05:56, 12.31s/it]
infer : od-8220                                             |  69%|######9   || 1387/2000 [4:44:43<2:05:44, 12.31s/it]
infer : od-8220                                             |  69%|######9   || 1388/2000 [4:44:55<2:05:30, 12.30s/it]
infer : od-8220                                             |  69%|######9   || 1389/2000 [4:45:08<2:05:15, 12.30s/it]
infer : od-8220                                             |  70%|######9   || 1390/2000 [4:45:20<2:05:05, 12.30s/it]
infer : od-8220                                             |  70%|######9   || 1391/2000 [4:45:32<2:04:54, 12.31s/it]
infer : od-8220                                             |  70%|######9   || 1392/2000 [4:45:44<2:04:42, 12.31s/it]
infer : od-8220                                             |  70%|######9   || 1393/2000 [4:45:57<2:04:29, 12.31s/it]
infer : od-8220                                             |  70%|######9   || 1394/2000 [4:46:09<2:04:14, 12.30s/it]
infer : od-8220                                             |  70%|######9   || 1395/2000 [4:46:21<2:04:02, 12.30s/it]
infer : od-8220                                             |  70%|######9   || 1396/2000 [4:46:34<2:03:50, 12.30s/it]
infer : od-8220                                             |  70%|######9   || 1397/2000 [4:46:46<2:03:41, 12.31s/it]
infer : od-8220                                             |  70%|######9   || 1398/2000 [4:46:58<2:03:29, 12.31s/it]
infer : od-8220                                             |  70%|######9   || 1399/2000 [4:47:11<2:03:15, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1400/2000 [4:47:23<2:03:00, 12.30s/it]
infer : od-8220                                             |  70%|#######   || 1401/2000 [4:47:35<2:02:48, 12.30s/it]
infer : od-8220                                             |  70%|#######   || 1402/2000 [4:47:47<2:02:37, 12.30s/it]
infer : od-8220                                             |  70%|#######   || 1403/2000 [4:48:00<2:02:27, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1404/2000 [4:48:12<2:02:16, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1405/2000 [4:48:24<2:02:03, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1406/2000 [4:48:37<2:01:48, 12.30s/it]
infer : od-8220                                             |  70%|#######   || 1407/2000 [4:48:49<2:01:33, 12.30s/it]
infer : od-8220                                             |  70%|#######   || 1408/2000 [4:49:01<2:01:25, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1409/2000 [4:49:14<2:01:14, 12.31s/it]
infer : od-8220                                             |  70%|#######   || 1410/2000 [4:49:26<2:01:02, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1411/2000 [4:49:38<2:00:48, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1412/2000 [4:49:51<2:00:34, 12.30s/it]
infer : od-8220                                             |  71%|#######   || 1413/2000 [4:50:03<2:00:20, 12.30s/it]
infer : od-8220                                             |  71%|#######   || 1414/2000 [4:50:15<2:00:12, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1415/2000 [4:50:27<1:59:58, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1416/2000 [4:50:40<1:59:51, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1417/2000 [4:50:52<1:59:36, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1418/2000 [4:51:04<1:59:21, 12.31s/it]
infer : od-8220                                             |  71%|#######   || 1419/2000 [4:51:17<1:59:10, 12.31s/it]
infer : od-8220                                             |  71%|#######1  || 1420/2000 [4:51:29<1:59:01, 12.31s/it]
infer : od-8220                                             |  71%|#######1  || 1421/2000 [4:51:41<1:58:52, 12.32s/it]
infer : od-8220                                             |  71%|#######1  || 1422/2000 [4:51:54<1:58:40, 12.32s/it]
infer : od-8220                                             |  71%|#######1  || 1423/2000 [4:52:06<1:58:24, 12.31s/it]
infer : od-8220                                             |  71%|#######1  || 1424/2000 [4:52:18<1:58:11, 12.31s/it]
infer : od-8220                                             |  71%|#######1  || 1425/2000 [4:52:31<1:58:00, 12.31s/it]
infer : od-8220                                             |  71%|#######1  || 1426/2000 [4:52:43<1:57:51, 12.32s/it]
infer : od-8220                                             |  71%|#######1  || 1427/2000 [4:52:55<1:57:39, 12.32s/it]
infer : od-8220                                             |  71%|#######1  || 1428/2000 [4:53:08<1:57:26, 12.32s/it]
infer : od-8220                                             |  71%|#######1  || 1429/2000 [4:53:20<1:57:08, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1430/2000 [4:53:32<1:56:54, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1431/2000 [4:53:44<1:56:43, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1432/2000 [4:53:57<1:56:33, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1433/2000 [4:54:09<1:56:23, 12.32s/it]
infer : od-8220                                             |  72%|#######1  || 1434/2000 [4:54:21<1:56:09, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1435/2000 [4:54:34<1:55:56, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1436/2000 [4:54:46<1:55:43, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1437/2000 [4:54:58<1:55:31, 12.31s/it]
infer : od-8220                                             |  72%|#######1  || 1438/2000 [4:55:11<1:55:21, 12.32s/it]
infer : od-8220                                             |  72%|#######1  || 1439/2000 [4:55:23<1:55:06, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1440/2000 [4:55:35<1:54:54, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1441/2000 [4:55:48<1:54:44, 12.32s/it]
infer : od-8220                                             |  72%|#######2  || 1442/2000 [4:56:00<1:54:31, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1443/2000 [4:56:12<1:54:20, 12.32s/it]
infer : od-8220                                             |  72%|#######2  || 1444/2000 [4:56:25<1:54:06, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1445/2000 [4:56:37<1:53:55, 12.32s/it]
infer : od-8220                                             |  72%|#######2  || 1446/2000 [4:56:49<1:53:44, 12.32s/it]
infer : od-8220                                             |  72%|#######2  || 1447/2000 [4:57:02<1:53:30, 12.32s/it]
infer : od-8220                                             |  72%|#######2  || 1448/2000 [4:57:14<1:53:15, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1449/2000 [4:57:26<1:53:03, 12.31s/it]
infer : od-8220                                             |  72%|#######2  || 1450/2000 [4:57:38<1:52:55, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1451/2000 [4:57:51<1:52:45, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1452/2000 [4:58:03<1:52:32, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1453/2000 [4:58:15<1:52:17, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1454/2000 [4:58:28<1:52:03, 12.31s/it]
infer : od-8220                                             |  73%|#######2  || 1455/2000 [4:58:40<1:51:52, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1456/2000 [4:58:52<1:51:42, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1457/2000 [4:59:05<1:51:28, 12.32s/it]
infer : od-8220                                             |  73%|#######2  || 1458/2000 [4:59:17<1:51:14, 12.31s/it]
infer : od-8220                                             |  73%|#######2  || 1459/2000 [4:59:29<1:51:00, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1460/2000 [4:59:42<1:50:47, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1461/2000 [4:59:54<1:50:37, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1462/2000 [5:00:06<1:50:25, 12.32s/it]
infer : od-8220                                             |  73%|#######3  || 1463/2000 [5:00:19<1:50:10, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1464/2000 [5:00:31<1:49:57, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1465/2000 [5:00:43<1:49:46, 12.31s/it]
infer : od-8220                                             |  73%|#######3  || 1466/2000 [5:00:56<1:49:40, 12.32s/it]
infer : od-8220                                             |  73%|#######3  || 1467/2000 [5:01:08<1:49:30, 12.33s/it]
infer : od-8220                                             |  73%|#######3  || 1468/2000 [5:01:20<1:49:15, 12.32s/it]
infer : od-8220                                             |  73%|#######3  || 1469/2000 [5:01:33<1:49:02, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1470/2000 [5:01:45<1:48:49, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1471/2000 [5:01:57<1:48:36, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1472/2000 [5:02:09<1:48:25, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1473/2000 [5:02:22<1:48:15, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1474/2000 [5:02:34<1:48:03, 12.33s/it]
infer : od-8220                                             |  74%|#######3  || 1475/2000 [5:02:46<1:47:49, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1476/2000 [5:02:59<1:47:34, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1477/2000 [5:03:11<1:47:20, 12.31s/it]
infer : od-8220                                             |  74%|#######3  || 1478/2000 [5:03:23<1:47:09, 12.32s/it]
infer : od-8220                                             |  74%|#######3  || 1479/2000 [5:03:36<1:46:59, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1480/2000 [5:03:48<1:46:48, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1481/2000 [5:04:00<1:46:36, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1482/2000 [5:04:13<1:46:23, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1483/2000 [5:04:25<1:46:07, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1484/2000 [5:04:37<1:45:57, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1485/2000 [5:04:50<1:45:51, 12.33s/it]
infer : od-8220                                             |  74%|#######4  || 1486/2000 [5:05:02<1:45:35, 12.33s/it]
infer : od-8220                                             |  74%|#######4  || 1487/2000 [5:05:14<1:45:19, 12.32s/it]
infer : od-8220                                             |  74%|#######4  || 1488/2000 [5:05:27<1:45:03, 12.31s/it]
infer : od-8220                                             |  74%|#######4  || 1489/2000 [5:05:39<1:44:49, 12.31s/it]
infer : od-8220                                             |  74%|#######4  || 1490/2000 [5:05:51<1:44:39, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1491/2000 [5:06:04<1:44:29, 12.32s/it]
infer : od-8220                                             |  75%|#######4  || 1492/2000 [5:06:16<1:44:14, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1493/2000 [5:06:28<1:43:59, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1494/2000 [5:06:40<1:43:47, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1495/2000 [5:06:53<1:43:36, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1496/2000 [5:07:05<1:43:26, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1497/2000 [5:07:17<1:43:13, 12.31s/it]
infer : od-8220                                             |  75%|#######4  || 1498/2000 [5:07:30<1:43:02, 12.32s/it]
infer : od-8220                                             |  75%|#######4  || 1499/2000 [5:07:42<1:42:47, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1500/2000 [5:07:54<1:42:33, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1501/2000 [5:08:07<1:42:22, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1502/2000 [5:08:19<1:42:12, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1503/2000 [5:08:31<1:42:00, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1504/2000 [5:08:44<1:41:48, 12.32s/it]
infer : od-8220                                             |  75%|#######5  || 1505/2000 [5:08:56<1:41:33, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1506/2000 [5:09:08<1:41:19, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1507/2000 [5:09:20<1:41:08, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1508/2000 [5:09:33<1:40:58, 12.31s/it]
infer : od-8220                                             |  75%|#######5  || 1509/2000 [5:09:45<1:40:49, 12.32s/it]
infer : od-8220                                             |  76%|#######5  || 1510/2000 [5:09:57<1:40:37, 12.32s/it]
infer : od-8220                                             |  76%|#######5  || 1511/2000 [5:10:10<1:40:21, 12.31s/it]
infer : od-8220                                             |  76%|#######5  || 1512/2000 [5:10:22<1:40:08, 12.31s/it]
infer : od-8220                                             |  76%|#######5  || 1513/2000 [5:10:34<1:39:56, 12.31s/it]
infer : od-8220                                             |  76%|#######5  || 1514/2000 [5:10:47<1:39:47, 12.32s/it]
infer : od-8220                                             |  76%|#######5  || 1515/2000 [5:10:59<1:39:39, 12.33s/it]
infer : od-8220                                             |  76%|#######5  || 1516/2000 [5:11:11<1:39:27, 12.33s/it]
infer : od-8220                                             |  76%|#######5  || 1517/2000 [5:11:24<1:39:11, 12.32s/it]
infer : od-8220                                             |  76%|#######5  || 1518/2000 [5:11:36<1:38:56, 12.32s/it]
infer : od-8220                                             |  76%|#######5  || 1519/2000 [5:11:48<1:38:44, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1520/2000 [5:12:01<1:38:32, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1521/2000 [5:12:13<1:38:18, 12.31s/it]
infer : od-8220                                             |  76%|#######6  || 1522/2000 [5:12:25<1:38:05, 12.31s/it]
infer : od-8220                                             |  76%|#######6  || 1523/2000 [5:12:38<1:37:55, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1524/2000 [5:12:50<1:37:41, 12.31s/it]
infer : od-8220                                             |  76%|#######6  || 1525/2000 [5:13:02<1:37:31, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1526/2000 [5:13:15<1:37:19, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1527/2000 [5:13:27<1:37:06, 12.32s/it]
infer : od-8220                                             |  76%|#######6  || 1528/2000 [5:13:39<1:36:50, 12.31s/it]
infer : od-8220                                             |  76%|#######6  || 1529/2000 [5:13:51<1:36:36, 12.31s/it]
infer : od-8220                                             |  76%|#######6  || 1530/2000 [5:14:04<1:36:24, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1531/2000 [5:14:16<1:36:13, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1532/2000 [5:14:28<1:36:00, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1533/2000 [5:14:41<1:35:48, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1534/2000 [5:14:53<1:35:34, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1535/2000 [5:15:05<1:35:21, 12.30s/it]
infer : od-8220                                             |  77%|#######6  || 1536/2000 [5:15:18<1:35:08, 12.30s/it]
infer : od-8220                                             |  77%|#######6  || 1537/2000 [5:15:30<1:35:00, 12.31s/it]
infer : od-8220                                             |  77%|#######6  || 1538/2000 [5:15:42<1:34:51, 12.32s/it]
infer : od-8220                                             |  77%|#######6  || 1539/2000 [5:15:55<1:34:40, 12.32s/it]
INFO:20250320-045522: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045622: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045722: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045822: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-045922: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050022: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050122: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050222: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050322: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050422: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050522: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050622: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050722: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050822: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-050922: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051022: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051122: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051223: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051323: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051423: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051523: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051623: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051723: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051823: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-051923: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052023: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052123: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052223: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052323: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052423: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052523: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052623: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052723: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052823: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-052923: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053023: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053123: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053223: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053323: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053423: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053523: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053623: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053723: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053823: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-053923: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054023: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054123: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054223: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054323: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054423: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054523: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054623: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054723: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054823: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-054923: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055023: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055123: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055223: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055323: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055423: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055523: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055624: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055724: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  77%|#######7  || 1540/2000 [5:16:07<1:34:25, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1541/2000 [5:16:19<1:34:10, 12.31s/it]
infer : od-8220                                             |  77%|#######7  || 1542/2000 [5:16:32<1:33:58, 12.31s/it]
infer : od-8220                                             |  77%|#######7  || 1543/2000 [5:16:44<1:33:49, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1544/2000 [5:16:56<1:33:37, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1545/2000 [5:17:08<1:33:24, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1546/2000 [5:17:21<1:33:11, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1547/2000 [5:17:33<1:32:59, 12.32s/it]
infer : od-8220                                             |  77%|#######7  || 1548/2000 [5:17:45<1:32:45, 12.31s/it]
infer : od-8220                                             |  77%|#######7  || 1549/2000 [5:17:58<1:32:33, 12.31s/it]
infer : od-8220                                             |  78%|#######7  || 1550/2000 [5:18:10<1:32:20, 12.31s/it]
infer : od-8220                                             |  78%|#######7  || 1551/2000 [5:18:22<1:32:08, 12.31s/it]
infer : od-8220                                             |  78%|#######7  || 1552/2000 [5:18:35<1:31:59, 12.32s/it]
infer : od-8220                                             |  78%|#######7  || 1553/2000 [5:18:47<1:31:46, 12.32s/it]
infer : od-8220                                             |  78%|#######7  || 1554/2000 [5:18:59<1:31:34, 12.32s/it]
infer : od-8220                                             |  78%|#######7  || 1555/2000 [5:19:12<1:31:22, 12.32s/it]
infer : od-8220                                             |  78%|#######7  || 1556/2000 [5:19:24<1:31:08, 12.32s/it]
infer : od-8220                                             |  78%|#######7  || 1557/2000 [5:19:36<1:30:54, 12.31s/it]
infer : od-8220                                             |  78%|#######7  || 1558/2000 [5:19:49<1:30:42, 12.31s/it]
infer : od-8220                                             |  78%|#######7  || 1559/2000 [5:20:01<1:30:29, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1560/2000 [5:20:13<1:30:17, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1561/2000 [5:20:26<1:30:03, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1562/2000 [5:20:38<1:29:51, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1563/2000 [5:20:50<1:29:39, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1564/2000 [5:21:02<1:29:24, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1565/2000 [5:21:15<1:29:11, 12.30s/it]
infer : od-8220                                             |  78%|#######8  || 1566/2000 [5:21:27<1:29:00, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1567/2000 [5:21:39<1:28:52, 12.32s/it]
infer : od-8220                                             |  78%|#######8  || 1568/2000 [5:21:52<1:28:39, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1569/2000 [5:22:04<1:28:24, 12.31s/it]
infer : od-8220                                             |  78%|#######8  || 1570/2000 [5:22:16<1:28:13, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1571/2000 [5:22:29<1:28:01, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1572/2000 [5:22:41<1:27:49, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1573/2000 [5:22:53<1:27:40, 12.32s/it]
infer : od-8220                                             |  79%|#######8  || 1574/2000 [5:23:06<1:27:26, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1575/2000 [5:23:18<1:27:11, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1576/2000 [5:23:30<1:26:57, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1577/2000 [5:23:42<1:26:44, 12.30s/it]
infer : od-8220                                             |  79%|#######8  || 1578/2000 [5:23:55<1:26:35, 12.31s/it]
infer : od-8220                                             |  79%|#######8  || 1579/2000 [5:24:07<1:26:23, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1580/2000 [5:24:19<1:26:08, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1581/2000 [5:24:32<1:25:55, 12.30s/it]
infer : od-8220                                             |  79%|#######9  || 1582/2000 [5:24:44<1:25:40, 12.30s/it]
infer : od-8220                                             |  79%|#######9  || 1583/2000 [5:24:56<1:25:31, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1584/2000 [5:25:09<1:25:22, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1585/2000 [5:25:21<1:25:10, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1586/2000 [5:25:33<1:24:58, 12.32s/it]
infer : od-8220                                             |  79%|#######9  || 1587/2000 [5:25:46<1:24:45, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1588/2000 [5:25:58<1:24:30, 12.31s/it]
infer : od-8220                                             |  79%|#######9  || 1589/2000 [5:26:10<1:24:20, 12.31s/it]
infer : od-8220                                             |  80%|#######9  || 1590/2000 [5:26:23<1:24:09, 12.32s/it]
infer : od-8220                                             |  80%|#######9  || 1591/2000 [5:26:35<1:23:54, 12.31s/it]
infer : od-8220                                             |  80%|#######9  || 1592/2000 [5:26:47<1:23:40, 12.31s/it]
infer : od-8220                                             |  80%|#######9  || 1593/2000 [5:26:59<1:23:26, 12.30s/it]
infer : od-8220                                             |  80%|#######9  || 1594/2000 [5:27:12<1:23:14, 12.30s/it]
infer : od-8220                                             |  80%|#######9  || 1595/2000 [5:27:24<1:23:02, 12.30s/it]
infer : od-8220                                             |  80%|#######9  || 1596/2000 [5:27:36<1:22:53, 12.31s/it]
infer : od-8220                                             |  80%|#######9  || 1597/2000 [5:27:49<1:22:47, 12.33s/it]
infer : od-8220                                             |  80%|#######9  || 1598/2000 [5:28:01<1:22:35, 12.33s/it]
infer : od-8220                                             |  80%|#######9  || 1599/2000 [5:28:13<1:22:22, 12.32s/it]
infer : od-8220                                             |  80%|########  || 1600/2000 [5:28:26<1:22:11, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1601/2000 [5:28:38<1:22:01, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1602/2000 [5:28:50<1:21:48, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1603/2000 [5:29:03<1:21:35, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1604/2000 [5:29:15<1:21:19, 12.32s/it]
infer : od-8220                                             |  80%|########  || 1605/2000 [5:29:27<1:21:05, 12.32s/it]
infer : od-8220                                             |  80%|########  || 1606/2000 [5:29:40<1:20:54, 12.32s/it]
infer : od-8220                                             |  80%|########  || 1607/2000 [5:29:52<1:20:43, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1608/2000 [5:30:04<1:20:32, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1609/2000 [5:30:17<1:20:19, 12.33s/it]
infer : od-8220                                             |  80%|########  || 1610/2000 [5:30:29<1:20:02, 12.31s/it]
infer : od-8220                                             |  81%|########  || 1611/2000 [5:30:41<1:19:51, 12.32s/it]
infer : od-8220                                             |  81%|########  || 1612/2000 [5:30:54<1:19:46, 12.34s/it]
infer : od-8220                                             |  81%|########  || 1613/2000 [5:31:06<1:19:36, 12.34s/it]
infer : od-8220                                             |  81%|########  || 1614/2000 [5:31:18<1:19:25, 12.35s/it]
infer : od-8220                                             |  81%|########  || 1615/2000 [5:31:31<1:19:12, 12.35s/it]
infer : od-8220                                             |  81%|########  || 1616/2000 [5:31:43<1:18:56, 12.33s/it]
infer : od-8220                                             |  81%|########  || 1617/2000 [5:31:55<1:18:41, 12.33s/it]
infer : od-8220                                             |  81%|########  || 1618/2000 [5:32:08<1:18:30, 12.33s/it]
infer : od-8220                                             |  81%|########  || 1619/2000 [5:32:20<1:18:17, 12.33s/it]
infer : od-8220                                             |  81%|########1 || 1620/2000 [5:32:32<1:18:06, 12.33s/it]
infer : od-8220                                             |  81%|########1 || 1621/2000 [5:32:45<1:17:52, 12.33s/it]
infer : od-8220                                             |  81%|########1 || 1622/2000 [5:32:57<1:17:38, 12.32s/it]
infer : od-8220                                             |  81%|########1 || 1623/2000 [5:33:09<1:17:24, 12.32s/it]
infer : od-8220                                             |  81%|########1 || 1624/2000 [5:33:22<1:17:11, 12.32s/it]
infer : od-8220                                             |  81%|########1 || 1625/2000 [5:33:34<1:17:01, 12.32s/it]
infer : od-8220                                             |  81%|########1 || 1626/2000 [5:33:46<1:16:53, 12.34s/it]
infer : od-8220                                             |  81%|########1 || 1627/2000 [5:33:59<1:16:40, 12.33s/it]
infer : od-8220                                             |  81%|########1 || 1628/2000 [5:34:11<1:16:27, 12.33s/it]
infer : od-8220                                             |  81%|########1 || 1629/2000 [5:34:23<1:16:14, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1630/2000 [5:34:36<1:16:01, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1631/2000 [5:34:48<1:15:49, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1632/2000 [5:35:00<1:15:41, 12.34s/it]
infer : od-8220                                             |  82%|########1 || 1633/2000 [5:35:13<1:15:25, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1634/2000 [5:35:25<1:15:10, 12.32s/it]
infer : od-8220                                             |  82%|########1 || 1635/2000 [5:35:37<1:14:58, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1636/2000 [5:35:50<1:14:47, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1637/2000 [5:36:02<1:14:34, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1638/2000 [5:36:14<1:14:21, 12.33s/it]
infer : od-8220                                             |  82%|########1 || 1639/2000 [5:36:26<1:14:08, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1640/2000 [5:36:39<1:13:54, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1641/2000 [5:36:51<1:13:41, 12.31s/it]
infer : od-8220                                             |  82%|########2 || 1642/2000 [5:37:03<1:13:28, 12.31s/it]
infer : od-8220                                             |  82%|########2 || 1643/2000 [5:37:16<1:13:17, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1644/2000 [5:37:28<1:13:06, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1645/2000 [5:37:40<1:12:54, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1646/2000 [5:37:53<1:12:41, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1647/2000 [5:38:05<1:12:28, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1648/2000 [5:38:17<1:12:16, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1649/2000 [5:38:30<1:12:04, 12.32s/it]
infer : od-8220                                             |  82%|########2 || 1650/2000 [5:38:42<1:11:50, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1651/2000 [5:38:54<1:11:40, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1652/2000 [5:39:07<1:11:28, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1653/2000 [5:39:19<1:11:16, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1654/2000 [5:39:31<1:11:04, 12.33s/it]
infer : od-8220                                             |  83%|########2 || 1655/2000 [5:39:44<1:10:53, 12.33s/it]
infer : od-8220                                             |  83%|########2 || 1656/2000 [5:39:56<1:10:39, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1657/2000 [5:40:08<1:10:26, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1658/2000 [5:40:21<1:10:13, 12.32s/it]
infer : od-8220                                             |  83%|########2 || 1659/2000 [5:40:33<1:10:07, 12.34s/it]
infer : od-8220                                             |  83%|########2 || 1660/2000 [5:40:45<1:09:58, 12.35s/it]
infer : od-8220                                             |  83%|########3 || 1661/2000 [5:40:58<1:09:44, 12.34s/it]
infer : od-8220                                             |  83%|########3 || 1662/2000 [5:41:10<1:09:29, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1663/2000 [5:41:22<1:09:14, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1664/2000 [5:41:35<1:09:02, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1665/2000 [5:41:47<1:08:50, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1666/2000 [5:41:59<1:08:38, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1667/2000 [5:42:12<1:08:28, 12.34s/it]
infer : od-8220                                             |  83%|########3 || 1668/2000 [5:42:24<1:08:14, 12.33s/it]
infer : od-8220                                             |  83%|########3 || 1669/2000 [5:42:36<1:08:03, 12.34s/it]
infer : od-8220                                             |  84%|########3 || 1670/2000 [5:42:49<1:07:50, 12.33s/it]
infer : od-8220                                             |  84%|########3 || 1671/2000 [5:43:01<1:07:38, 12.34s/it]
infer : od-8220                                             |  84%|########3 || 1672/2000 [5:43:13<1:07:26, 12.34s/it]
infer : od-8220                                             |  84%|########3 || 1673/2000 [5:43:26<1:07:13, 12.33s/it]
infer : od-8220                                             |  84%|########3 || 1674/2000 [5:43:38<1:06:56, 12.32s/it]
infer : od-8220                                             |  84%|########3 || 1675/2000 [5:43:50<1:06:43, 12.32s/it]
infer : od-8220                                             |  84%|########3 || 1676/2000 [5:44:03<1:06:32, 12.32s/it]
infer : od-8220                                             |  84%|########3 || 1677/2000 [5:44:15<1:06:21, 12.33s/it]
infer : od-8220                                             |  84%|########3 || 1678/2000 [5:44:27<1:06:09, 12.33s/it]
infer : od-8220                                             |  84%|########3 || 1679/2000 [5:44:40<1:05:58, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1680/2000 [5:44:52<1:05:45, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1681/2000 [5:45:04<1:05:31, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1682/2000 [5:45:17<1:05:20, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1683/2000 [5:45:29<1:05:07, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1684/2000 [5:45:41<1:04:57, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1685/2000 [5:45:54<1:04:44, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1686/2000 [5:46:06<1:04:30, 12.33s/it]
infer : od-8220                                             |  84%|########4 || 1687/2000 [5:46:18<1:04:17, 12.32s/it]
infer : od-8220                                             |  84%|########4 || 1688/2000 [5:46:31<1:04:05, 12.32s/it]
infer : od-8220                                             |  84%|########4 || 1689/2000 [5:46:43<1:03:52, 12.32s/it]
infer : od-8220                                             |  84%|########4 || 1690/2000 [5:46:55<1:03:41, 12.33s/it]
infer : od-8220                                             |  85%|########4 || 1691/2000 [5:47:07<1:03:26, 12.32s/it]
infer : od-8220                                             |  85%|########4 || 1692/2000 [5:47:20<1:03:14, 12.32s/it]
infer : od-8220                                             |  85%|########4 || 1693/2000 [5:47:32<1:03:02, 12.32s/it]
infer : od-8220                                             |  85%|########4 || 1694/2000 [5:47:44<1:02:50, 12.32s/it]
infer : od-8220                                             |  85%|########4 || 1695/2000 [5:47:57<1:02:40, 12.33s/it]
infer : od-8220                                             |  85%|########4 || 1696/2000 [5:48:09<1:02:27, 12.33s/it]
infer : od-8220                                             |  85%|########4 || 1697/2000 [5:48:21<1:02:12, 12.32s/it]
infer : od-8220                                             |  85%|########4 || 1698/2000 [5:48:34<1:02:02, 12.33s/it]
infer : od-8220                                             |  85%|########4 || 1699/2000 [5:48:46<1:01:51, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1700/2000 [5:48:58<1:01:40, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1701/2000 [5:49:11<1:01:28, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1702/2000 [5:49:23<1:01:14, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1703/2000 [5:49:35<1:01:00, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1704/2000 [5:49:48<1:00:48, 12.32s/it]
infer : od-8220                                             |  85%|########5 || 1705/2000 [5:50:00<1:00:37, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1706/2000 [5:50:12<1:00:25, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1707/2000 [5:50:25<1:00:13, 12.33s/it]
infer : od-8220                                             |  85%|########5 || 1708/2000 [5:50:37<1:00:03, 12.34s/it]
infer : od-8220                                             |  85%|########5 || 1709/2000 [5:50:49<59:48, 12.33s/it]  
infer : od-8220                                             |  86%|########5 || 1710/2000 [5:51:02<59:35, 12.33s/it]
infer : od-8220                                             |  86%|########5 || 1711/2000 [5:51:14<59:22, 12.33s/it]
infer : od-8220                                             |  86%|########5 || 1712/2000 [5:51:26<59:10, 12.33s/it]
infer : od-8220                                             |  86%|########5 || 1713/2000 [5:51:39<58:58, 12.33s/it]
infer : od-8220                                             |  86%|########5 || 1714/2000 [5:51:51<58:45, 12.33s/it]
infer : od-8220                                             |  86%|########5 || 1715/2000 [5:52:03<58:31, 12.32s/it]
infer : od-8220                                             |  86%|########5 || 1716/2000 [5:52:16<58:17, 12.32s/it]
infer : od-8220                                             |  86%|########5 || 1717/2000 [5:52:28<58:04, 12.31s/it]
infer : od-8220                                             |  86%|########5 || 1718/2000 [5:52:40<57:54, 12.32s/it]
infer : od-8220                                             |  86%|########5 || 1719/2000 [5:52:53<57:43, 12.33s/it]
infer : od-8220                                             |  86%|########6 || 1720/2000 [5:53:05<57:30, 12.32s/it]
infer : od-8220                                             |  86%|########6 || 1721/2000 [5:53:17<57:16, 12.32s/it]
infer : od-8220                                             |  86%|########6 || 1722/2000 [5:53:30<57:04, 12.32s/it]
infer : od-8220                                             |  86%|########6 || 1723/2000 [5:53:42<56:51, 12.31s/it]
infer : od-8220                                             |  86%|########6 || 1724/2000 [5:53:54<56:38, 12.31s/it]
infer : od-8220                                             |  86%|########6 || 1725/2000 [5:54:06<56:25, 12.31s/it]
infer : od-8220                                             |  86%|########6 || 1726/2000 [5:54:19<56:11, 12.30s/it]
infer : od-8220                                             |  86%|########6 || 1727/2000 [5:54:31<55:58, 12.30s/it]
infer : od-8220                                             |  86%|########6 || 1728/2000 [5:54:43<55:44, 12.30s/it]
infer : od-8220                                             |  86%|########6 || 1729/2000 [5:54:56<55:34, 12.30s/it]
infer : od-8220                                             |  86%|########6 || 1730/2000 [5:55:08<55:20, 12.30s/it]
infer : od-8220                                             |  87%|########6 || 1731/2000 [5:55:20<55:09, 12.30s/it]
infer : od-8220                                             |  87%|########6 || 1732/2000 [5:55:33<54:56, 12.30s/it]
infer : od-8220                                             |  87%|########6 || 1733/2000 [5:55:45<54:43, 12.30s/it]
infer : od-8220                                             |  87%|########6 || 1734/2000 [5:55:57<54:33, 12.31s/it]
infer : od-8220                                             |  87%|########6 || 1735/2000 [5:56:09<54:20, 12.30s/it]
infer : od-8220                                             |  87%|########6 || 1736/2000 [5:56:22<54:09, 12.31s/it]
infer : od-8220                                             |  87%|########6 || 1737/2000 [5:56:34<53:57, 12.31s/it]
infer : od-8220                                             |  87%|########6 || 1738/2000 [5:56:46<53:44, 12.31s/it]
infer : od-8220                                             |  87%|########6 || 1739/2000 [5:56:59<53:30, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1740/2000 [5:57:11<53:18, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1741/2000 [5:57:23<53:07, 12.31s/it]
infer : od-8220                                             |  87%|########7 || 1742/2000 [5:57:36<52:54, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1743/2000 [5:57:48<52:41, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1744/2000 [5:58:00<52:29, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1745/2000 [5:58:13<52:15, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1746/2000 [5:58:25<52:03, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1747/2000 [5:58:37<51:51, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1748/2000 [5:58:49<51:40, 12.30s/it]
infer : od-8220                                             |  87%|########7 || 1749/2000 [5:59:02<51:28, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1750/2000 [5:59:14<51:14, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1751/2000 [5:59:26<51:02, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1752/2000 [5:59:39<50:51, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1753/2000 [5:59:51<50:39, 12.31s/it]
infer : od-8220                                             |  88%|########7 || 1754/2000 [6:00:03<50:27, 12.31s/it]
infer : od-8220                                             |  88%|########7 || 1755/2000 [6:00:16<50:14, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1756/2000 [6:00:28<50:00, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1757/2000 [6:00:40<49:48, 12.30s/it]
infer : od-8220                                             |  88%|########7 || 1758/2000 [6:00:52<49:39, 12.31s/it]
infer : od-8220                                             |  88%|########7 || 1759/2000 [6:01:05<49:28, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1760/2000 [6:01:17<49:16, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1761/2000 [6:01:29<49:02, 12.31s/it]
infer : od-8220                                             |  88%|########8 || 1762/2000 [6:01:42<48:49, 12.31s/it]
infer : od-8220                                             |  88%|########8 || 1763/2000 [6:01:54<48:37, 12.31s/it]
infer : od-8220                                             |  88%|########8 || 1764/2000 [6:02:06<48:25, 12.31s/it]
infer : od-8220                                             |  88%|########8 || 1765/2000 [6:02:19<48:15, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1766/2000 [6:02:31<48:03, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1767/2000 [6:02:43<47:50, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1768/2000 [6:02:56<47:37, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1769/2000 [6:03:08<47:25, 12.32s/it]
infer : od-8220                                             |  88%|########8 || 1770/2000 [6:03:20<47:11, 12.31s/it]
infer : od-8220                                             |  89%|########8 || 1771/2000 [6:03:33<47:00, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1772/2000 [6:03:45<46:47, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1773/2000 [6:03:57<46:35, 12.31s/it]
infer : od-8220                                             |  89%|########8 || 1774/2000 [6:04:10<46:23, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1775/2000 [6:04:22<46:12, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1776/2000 [6:04:34<46:01, 12.33s/it]
infer : od-8220                                             |  89%|########8 || 1777/2000 [6:04:47<45:47, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1778/2000 [6:04:59<45:33, 12.32s/it]
infer : od-8220                                             |  89%|########8 || 1779/2000 [6:05:11<45:20, 12.31s/it]
infer : od-8220                                             |  89%|########9 || 1780/2000 [6:05:23<45:09, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1781/2000 [6:05:36<44:57, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1782/2000 [6:05:48<44:45, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1783/2000 [6:06:00<44:34, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1784/2000 [6:06:13<44:20, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1785/2000 [6:06:25<44:06, 12.31s/it]
infer : od-8220                                             |  89%|########9 || 1786/2000 [6:06:37<43:54, 12.31s/it]
infer : od-8220                                             |  89%|########9 || 1787/2000 [6:06:50<43:43, 12.31s/it]
infer : od-8220                                             |  89%|########9 || 1788/2000 [6:07:02<43:30, 12.32s/it]
infer : od-8220                                             |  89%|########9 || 1789/2000 [6:07:14<43:17, 12.31s/it]
infer : od-8220                                             |  90%|########9 || 1790/2000 [6:07:27<43:02, 12.30s/it]
infer : od-8220                                             |  90%|########9 || 1791/2000 [6:07:39<42:48, 12.29s/it]
infer : od-8220                                             |  90%|########9 || 1792/2000 [6:07:51<42:38, 12.30s/it]
infer : od-8220                                             |  90%|########9 || 1793/2000 [6:08:03<42:25, 12.30s/it]
infer : od-8220                                             |  90%|########9 || 1794/2000 [6:08:16<42:12, 12.30s/it]
infer : od-8220                                             |  90%|########9 || 1795/2000 [6:08:28<42:00, 12.29s/it]
infer : od-8220                                             |  90%|########9 || 1796/2000 [6:08:40<41:47, 12.29s/it]
infer : od-8220                                             |  90%|########9 || 1797/2000 [6:08:53<41:35, 12.29s/it]
infer : od-8220                                             |  90%|########9 || 1798/2000 [6:09:05<41:23, 12.29s/it]
infer : od-8220                                             |  90%|########9 || 1799/2000 [6:09:17<41:10, 12.29s/it]
infer : od-8220                                             |  90%|######### || 1800/2000 [6:09:29<40:56, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1801/2000 [6:09:42<40:43, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1802/2000 [6:09:54<40:31, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1803/2000 [6:10:06<40:17, 12.27s/it]
infer : od-8220                                             |  90%|######### || 1804/2000 [6:10:19<40:05, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1805/2000 [6:10:31<39:54, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1806/2000 [6:10:43<39:43, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1807/2000 [6:10:55<39:30, 12.28s/it]
infer : od-8220                                             |  90%|######### || 1808/2000 [6:11:08<39:18, 12.29s/it]
infer : od-8220                                             |  90%|######### || 1809/2000 [6:11:20<39:04, 12.27s/it]
infer : od-8220                                             |  90%|######### || 1810/2000 [6:11:32<38:53, 12.28s/it]
infer : od-8220                                             |  91%|######### || 1811/2000 [6:11:45<38:43, 12.29s/it]
infer : od-8220                                             |  91%|######### || 1812/2000 [6:11:57<38:31, 12.30s/it]
infer : od-8220                                             |  91%|######### || 1813/2000 [6:12:09<38:17, 12.29s/it]
infer : od-8220                                             |  91%|######### || 1814/2000 [6:12:21<38:03, 12.28s/it]
infer : od-8220                                             |  91%|######### || 1815/2000 [6:12:34<37:51, 12.28s/it]
infer : od-8220                                             |  91%|######### || 1816/2000 [6:12:46<37:39, 12.28s/it]
infer : od-8220                                             |  91%|######### || 1817/2000 [6:12:58<37:26, 12.28s/it]
infer : od-8220                                             |  91%|######### || 1818/2000 [6:13:10<37:13, 12.27s/it]
infer : od-8220                                             |  91%|######### || 1819/2000 [6:13:23<37:01, 12.27s/it]
infer : od-8220                                             |  91%|#########1|| 1820/2000 [6:13:35<36:50, 12.28s/it]
infer : od-8220                                             |  91%|#########1|| 1821/2000 [6:13:47<36:38, 12.28s/it]
infer : od-8220                                             |  91%|#########1|| 1822/2000 [6:14:00<36:26, 12.28s/it]
infer : od-8220                                             |  91%|#########1|| 1823/2000 [6:14:12<36:14, 12.29s/it]
infer : od-8220                                             |  91%|#########1|| 1824/2000 [6:14:24<36:03, 12.29s/it]
infer : od-8220                                             |  91%|#########1|| 1825/2000 [6:14:37<35:52, 12.30s/it]
infer : od-8220                                             |  91%|#########1|| 1826/2000 [6:14:49<35:39, 12.30s/it]
infer : od-8220                                             |  91%|#########1|| 1827/2000 [6:15:01<35:28, 12.30s/it]
infer : od-8220                                             |  91%|#########1|| 1828/2000 [6:15:13<35:15, 12.30s/it]
infer : od-8220                                             |  91%|#########1|| 1829/2000 [6:15:26<35:03, 12.30s/it]
infer : od-8220                                             |  92%|#########1|| 1830/2000 [6:15:38<34:51, 12.30s/it]
infer : od-8220                                             |  92%|#########1|| 1831/2000 [6:15:50<34:39, 12.30s/it]
infer : od-8220                                             |  92%|#########1|| 1832/2000 [6:16:03<34:27, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1833/2000 [6:16:15<34:14, 12.30s/it]
infer : od-8220                                             |  92%|#########1|| 1834/2000 [6:16:27<34:02, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1835/2000 [6:16:40<33:50, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1836/2000 [6:16:52<33:38, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1837/2000 [6:17:04<33:26, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1838/2000 [6:17:17<33:13, 12.31s/it]
infer : od-8220                                             |  92%|#########1|| 1839/2000 [6:17:29<33:01, 12.31s/it]
infer : od-8220                                             |  92%|#########2|| 1840/2000 [6:17:41<32:51, 12.32s/it]
infer : od-8220                                             |  92%|#########2|| 1841/2000 [6:17:53<32:38, 12.32s/it]
infer : od-8220                                             |  92%|#########2|| 1842/2000 [6:18:06<32:25, 12.32s/it]
infer : od-8220                                             |  92%|#########2|| 1843/2000 [6:18:18<32:13, 12.31s/it]
infer : od-8220                                             |  92%|#########2|| 1844/2000 [6:18:30<31:59, 12.31s/it]
infer : od-8220                                             |  92%|#########2|| 1845/2000 [6:18:43<31:48, 12.32s/it]
infer : od-8220                                             |  92%|#########2|| 1846/2000 [6:18:55<31:36, 12.31s/it]
infer : od-8220                                             |  92%|#########2|| 1847/2000 [6:19:07<31:23, 12.31s/it]
INFO:20250320-055824: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-055924: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060024: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060124: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060224: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060324: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060424: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060524: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060624: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060724: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060824: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-060924: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061024: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061124: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061224: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061324: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061424: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061524: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061624: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061724: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061824: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-061924: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062024: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062124: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062224: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062324: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062424: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062524: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062624: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062724: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062824: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-062924: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063024: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063124: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063224: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063324: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063424: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063524: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063624: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063724: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063824: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-063924: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064025: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064125: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064225: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064325: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064425: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064525: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064625: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064725: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064825: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-064925: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065025: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065125: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065225: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065325: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065425: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065525: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065625: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065725: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065825: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-065925: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070025: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

infer : od-8220                                             |  92%|#########2|| 1848/2000 [6:19:20<31:10, 12.30s/it]
infer : od-8220                                             |  92%|#########2|| 1849/2000 [6:19:32<30:57, 12.30s/it]
infer : od-8220                                             |  92%|#########2|| 1850/2000 [6:19:44<30:45, 12.30s/it]
infer : od-8220                                             |  93%|#########2|| 1851/2000 [6:19:57<30:34, 12.31s/it]
infer : od-8220                                             |  93%|#########2|| 1852/2000 [6:20:09<30:23, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1853/2000 [6:20:21<30:11, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1854/2000 [6:20:34<29:59, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1855/2000 [6:20:46<29:45, 12.31s/it]
infer : od-8220                                             |  93%|#########2|| 1856/2000 [6:20:58<29:33, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1857/2000 [6:21:10<29:21, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1858/2000 [6:21:23<29:09, 12.32s/it]
infer : od-8220                                             |  93%|#########2|| 1859/2000 [6:21:35<28:56, 12.32s/it]
infer : od-8220                                             |  93%|#########3|| 1860/2000 [6:21:47<28:43, 12.31s/it]
infer : od-8220                                             |  93%|#########3|| 1861/2000 [6:22:00<28:30, 12.31s/it]
infer : od-8220                                             |  93%|#########3|| 1862/2000 [6:22:12<28:19, 12.31s/it]
infer : od-8220                                             |  93%|#########3|| 1863/2000 [6:22:24<28:06, 12.31s/it]
infer : od-8220                                             |  93%|#########3|| 1864/2000 [6:22:37<27:53, 12.31s/it]
infer : od-8220                                             |  93%|#########3|| 1865/2000 [6:22:49<27:40, 12.30s/it]
infer : od-8220                                             |  93%|#########3|| 1866/2000 [6:23:01<27:27, 12.30s/it]
infer : od-8220                                             |  93%|#########3|| 1867/2000 [6:23:14<27:15, 12.29s/it]
infer : od-8220                                             |  93%|#########3|| 1868/2000 [6:23:26<27:02, 12.29s/it]
infer : od-8220                                             |  93%|#########3|| 1869/2000 [6:23:38<26:51, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1870/2000 [6:23:50<26:39, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1871/2000 [6:24:03<26:26, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1872/2000 [6:24:15<26:13, 12.29s/it]
infer : od-8220                                             |  94%|#########3|| 1873/2000 [6:24:27<26:01, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1874/2000 [6:24:40<25:50, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1875/2000 [6:24:52<25:38, 12.31s/it]
infer : od-8220                                             |  94%|#########3|| 1876/2000 [6:25:04<25:25, 12.31s/it]
infer : od-8220                                             |  94%|#########3|| 1877/2000 [6:25:17<25:12, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1878/2000 [6:25:29<25:00, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1879/2000 [6:25:41<24:48, 12.30s/it]
infer : od-8220                                             |  94%|#########3|| 1880/2000 [6:25:53<24:37, 12.31s/it]
infer : od-8220                                             |  94%|#########4|| 1881/2000 [6:26:06<24:25, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1882/2000 [6:26:18<24:13, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1883/2000 [6:26:30<24:00, 12.31s/it]
infer : od-8220                                             |  94%|#########4|| 1884/2000 [6:26:43<23:48, 12.31s/it]
infer : od-8220                                             |  94%|#########4|| 1885/2000 [6:26:55<23:36, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1886/2000 [6:27:07<23:24, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1887/2000 [6:27:20<23:11, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1888/2000 [6:27:32<22:59, 12.31s/it]
infer : od-8220                                             |  94%|#########4|| 1889/2000 [6:27:44<22:47, 12.32s/it]
infer : od-8220                                             |  94%|#########4|| 1890/2000 [6:27:57<22:34, 12.31s/it]
infer : od-8220                                             |  95%|#########4|| 1891/2000 [6:28:09<22:21, 12.31s/it]
infer : od-8220                                             |  95%|#########4|| 1892/2000 [6:28:21<22:09, 12.31s/it]
infer : od-8220                                             |  95%|#########4|| 1893/2000 [6:28:33<21:55, 12.30s/it]
infer : od-8220                                             |  95%|#########4|| 1894/2000 [6:28:46<21:43, 12.30s/it]
infer : od-8220                                             |  95%|#########4|| 1895/2000 [6:28:58<21:31, 12.30s/it]
infer : od-8220                                             |  95%|#########4|| 1896/2000 [6:29:10<21:18, 12.29s/it]
infer : od-8220                                             |  95%|#########4|| 1897/2000 [6:29:23<21:06, 12.30s/it]
infer : od-8220                                             |  95%|#########4|| 1898/2000 [6:29:35<20:55, 12.31s/it]
infer : od-8220                                             |  95%|#########4|| 1899/2000 [6:29:47<20:43, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1900/2000 [6:30:00<20:30, 12.30s/it]
infer : od-8220                                             |  95%|#########5|| 1901/2000 [6:30:12<20:17, 12.30s/it]
infer : od-8220                                             |  95%|#########5|| 1902/2000 [6:30:24<20:05, 12.30s/it]
infer : od-8220                                             |  95%|#########5|| 1903/2000 [6:30:37<19:53, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1904/2000 [6:30:49<19:41, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1905/2000 [6:31:01<19:29, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1906/2000 [6:31:13<19:17, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1907/2000 [6:31:26<19:05, 12.31s/it]
infer : od-8220                                             |  95%|#########5|| 1908/2000 [6:31:38<18:53, 12.32s/it]
infer : od-8220                                             |  95%|#########5|| 1909/2000 [6:31:50<18:41, 12.32s/it]
infer : od-8220                                             |  96%|#########5|| 1910/2000 [6:32:03<18:28, 12.32s/it]
infer : od-8220                                             |  96%|#########5|| 1911/2000 [6:32:15<18:15, 12.31s/it]
infer : od-8220                                             |  96%|#########5|| 1912/2000 [6:32:27<18:03, 12.31s/it]
infer : od-8220                                             |  96%|#########5|| 1913/2000 [6:32:40<17:51, 12.32s/it]
infer : od-8220                                             |  96%|#########5|| 1914/2000 [6:32:52<17:39, 12.32s/it]
infer : od-8220                                             |  96%|#########5|| 1915/2000 [6:33:04<17:27, 12.32s/it]
infer : od-8220                                             |  96%|#########5|| 1916/2000 [6:33:17<17:14, 12.31s/it]
infer : od-8220                                             |  96%|#########5|| 1917/2000 [6:33:29<17:01, 12.31s/it]
infer : od-8220                                             |  96%|#########5|| 1918/2000 [6:33:41<16:49, 12.31s/it]
infer : od-8220                                             |  96%|#########5|| 1919/2000 [6:33:54<16:37, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1920/2000 [6:34:06<16:24, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1921/2000 [6:34:18<16:12, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1922/2000 [6:34:31<16:00, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1923/2000 [6:34:43<15:47, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1924/2000 [6:34:55<15:35, 12.30s/it]
infer : od-8220                                             |  96%|#########6|| 1925/2000 [6:35:07<15:22, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1926/2000 [6:35:20<15:10, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1927/2000 [6:35:32<14:58, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1928/2000 [6:35:44<14:46, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1929/2000 [6:35:57<14:33, 12.31s/it]
infer : od-8220                                             |  96%|#########6|| 1930/2000 [6:36:09<14:21, 12.30s/it]
infer : od-8220                                             |  97%|#########6|| 1931/2000 [6:36:21<14:09, 12.31s/it]
infer : od-8220                                             |  97%|#########6|| 1932/2000 [6:36:34<13:56, 12.30s/it]
infer : od-8220                                             |  97%|#########6|| 1933/2000 [6:36:46<13:44, 12.31s/it]
infer : od-8220                                             |  97%|#########6|| 1934/2000 [6:36:58<13:31, 12.30s/it]
infer : od-8220                                             |  97%|#########6|| 1935/2000 [6:37:10<13:19, 12.30s/it]
infer : od-8220                                             |  97%|#########6|| 1936/2000 [6:37:23<13:07, 12.30s/it]
infer : od-8220                                             |  97%|#########6|| 1937/2000 [6:37:35<12:55, 12.31s/it]
infer : od-8220                                             |  97%|#########6|| 1938/2000 [6:37:47<12:43, 12.31s/it]
infer : od-8220                                             |  97%|#########6|| 1939/2000 [6:38:00<12:30, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1940/2000 [6:38:12<12:17, 12.30s/it]
infer : od-8220                                             |  97%|#########7|| 1941/2000 [6:38:24<12:05, 12.30s/it]
infer : od-8220                                             |  97%|#########7|| 1942/2000 [6:38:37<11:53, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1943/2000 [6:38:49<11:41, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1944/2000 [6:39:01<11:29, 12.32s/it]
infer : od-8220                                             |  97%|#########7|| 1945/2000 [6:39:14<11:17, 12.32s/it]
infer : od-8220                                             |  97%|#########7|| 1946/2000 [6:39:26<11:04, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1947/2000 [6:39:38<10:52, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1948/2000 [6:39:50<10:40, 12.31s/it]
infer : od-8220                                             |  97%|#########7|| 1949/2000 [6:40:03<10:27, 12.31s/it]
infer : od-8220                                             |  98%|#########7|| 1950/2000 [6:40:15<10:15, 12.31s/it]
infer : od-8220                                             |  98%|#########7|| 1951/2000 [6:40:27<10:02, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1952/2000 [6:40:40<09:50, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1953/2000 [6:40:52<09:38, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1954/2000 [6:41:04<09:26, 12.31s/it]
infer : od-8220                                             |  98%|#########7|| 1955/2000 [6:41:17<09:13, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1956/2000 [6:41:29<09:01, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1957/2000 [6:41:41<08:49, 12.31s/it]
infer : od-8220                                             |  98%|#########7|| 1958/2000 [6:41:54<08:36, 12.30s/it]
infer : od-8220                                             |  98%|#########7|| 1959/2000 [6:42:06<08:24, 12.30s/it]
infer : od-8220                                             |  98%|#########8|| 1960/2000 [6:42:18<08:12, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1961/2000 [6:42:30<08:00, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1962/2000 [6:42:43<07:47, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1963/2000 [6:42:55<07:35, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1964/2000 [6:43:07<07:22, 12.30s/it]
infer : od-8220                                             |  98%|#########8|| 1965/2000 [6:43:20<07:10, 12.30s/it]
infer : od-8220                                             |  98%|#########8|| 1966/2000 [6:43:32<06:58, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1967/2000 [6:43:44<06:46, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1968/2000 [6:43:57<06:33, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1969/2000 [6:44:09<06:21, 12.31s/it]
infer : od-8220                                             |  98%|#########8|| 1970/2000 [6:44:21<06:09, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1971/2000 [6:44:34<05:57, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1972/2000 [6:44:46<05:44, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1973/2000 [6:44:58<05:32, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1974/2000 [6:45:10<05:20, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1975/2000 [6:45:23<05:07, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1976/2000 [6:45:35<04:55, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1977/2000 [6:45:47<04:43, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1978/2000 [6:46:00<04:30, 12.31s/it]
infer : od-8220                                             |  99%|#########8|| 1979/2000 [6:46:12<04:18, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1980/2000 [6:46:24<04:06, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1981/2000 [6:46:37<03:53, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1982/2000 [6:46:49<03:41, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1983/2000 [6:47:01<03:29, 12.31s/it]
infer : od-8220                                             |  99%|#########9|| 1984/2000 [6:47:14<03:16, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1985/2000 [6:47:26<03:04, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1986/2000 [6:47:38<02:52, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1987/2000 [6:47:50<02:39, 12.30s/it]
infer : od-8220                                             |  99%|#########9|| 1988/2000 [6:48:03<02:27, 12.31s/it]
infer : od-8220                                             |  99%|#########9|| 1989/2000 [6:48:15<02:15, 12.31s/it]
infer : od-8220                                             | 100%|#########9|| 1990/2000 [6:48:27<02:03, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1991/2000 [6:48:40<01:50, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1992/2000 [6:48:52<01:38, 12.29s/it]
infer : od-8220                                             | 100%|#########9|| 1993/2000 [6:49:04<01:26, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1994/2000 [6:49:17<01:13, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1995/2000 [6:49:29<01:01, 12.31s/it]
infer : od-8220                                             | 100%|#########9|| 1996/2000 [6:49:41<00:49, 12.31s/it]
infer : od-8220                                             | 100%|#########9|| 1997/2000 [6:49:53<00:36, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1998/2000 [6:50:06<00:24, 12.30s/it]
infer : od-8220                                             | 100%|#########9|| 1999/2000 [6:50:18<00:12, 12.30s/it]
infer : od-8220                                             | 100%|##########|| 2000/2000 [6:50:30<00:00, 12.31s/it]
infer : od-8220                                             | 100%|##########|| 2000/2000 [6:50:30<00:00, 12.32s/it]

INFO:20250320-073248: infer completed  - od-8220 - 24631 secLoading and preparing results...
DONE (t=0.03s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=1.99s).
Accumulating evaluation results...
DONE (t=0.31s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.150
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.389
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.072
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.096
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.183
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.249
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.125
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.190
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.190
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.136
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.221
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.285


SUCCESS:20250320-073250: benchmark results - {'infer_path': 'od-8220', 'accuracy_ap[.5:.95]%': 15.029691, 'accuracy_ap50%': 38.856026, 'num_subgraphs': 0, 'infer_time_core_ms': 12303.004554, 'infer_time_subgraph_ms': 0.0, 'ddr_transfer_mb': 0.0, 'perfsim_time_ms': 81.37782, 'perfsim_ddr_transfer_mb': 67.33, 'perfsim_gmacs': 13.3257}

INFO:20250320-070125: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070225: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070325: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070425: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070525: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070625: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070725: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070825: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-070925: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071025: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071125: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071225: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071325: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071425: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071525: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071625: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071725: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071825: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-071925: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072025: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072125: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072225: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072325: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072426: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072526: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072626: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072726: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072826: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-072926: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-073026: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-073126: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0

INFO:20250320-073226: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0


TASKS                                                       | 100%|██████████|| 1/1 [6:50:33<00:00, 24633.76s/it]
TASKS                                                       | 100%|██████████|| 1/1 [6:50:33<00:00, 24633.86s/it]


packaging artifacts to /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/pkg please wait...
SUCCESS:20250320-073252: finished packaging - /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/work/od-8220
Compiled model is at: /local_data/home/mattmak/edgeai-tensorlab/edgeai-modelmaker/data/projects/Shirley-20250319/run/20250319-174853/yolox_s_lite/compilation/AM62A/pkg/20250319-174853_yolox_s_lite_onnxrt_AM62A.tar.gz