我正在对一些科学计算项目进行网格搜索,并得到了一堆文件名中嵌入了参数的日志文件。
$ ls -l | wc -l
38210
$ ls -l | tail
-rw-rw-r-- 1 qys wheel 1070 Jul 20 15:01 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda0_kappa0.25.txt
-rw-rw-r-- 1 qys wheel 1070 Jul 20 15:00 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda0_kappa0.5.txt
-rw-rw-r-- 1 qys wheel 1070 Jul 20 15:01 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda0_kappa0.75.txt
-rw-rw-r-- 1 qys wheel 1070 Jul 20 15:00 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda0_kappa0.txt
-rw-rw-r-- 1 qys wheel 1070 Jul 20 15:01 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda0_kappa1.txt
-rw-rw-r-- 1 qys wheel 981 Jul 20 15:06 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa0.25.txt
-rw-rw-r-- 1 qys wheel 981 Jul 20 15:08 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa0.5.txt
-rw-rw-r-- 1 qys wheel 981 Jul 20 15:08 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa0.75.txt
-rw-rw-r-- 1 qys wheel 981 Jul 20 15:06 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa0.txt
-rw-rw-r-- 1 qys wheel 984 Jul 20 15:08 gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa1.txt
每个文件看起来都像
$ cat gmm_dim96_aelr0.01_ufactor100_K8_R8_lr1_lambda1_kappa1.txt
GMM: epoch 0: train loss 662699.9375, valid accuracy 80.03%
GMM: epoch 1: train loss 662553.75, valid accuracy 79.92%
GMM: epoch 2: train loss 662446.1875, valid accuracy 79.77%
GMM: epoch 3: train loss 662429.5, valid accuracy 79.82%
GMM: epoch 4: train loss 662404.5625, valid accuracy 79.83%
GMM: epoch 5: train loss 662395.0625, valid accuracy 79.85%
GMM: epoch 6: train loss 662381.25, valid accuracy 79.85%
GMM: epoch 7: train loss 662370.25, valid accuracy 79.83%
GMM: epoch 8: train loss 662362.875, valid accuracy 79.86%
GMM: epoch 9: train loss 662353.6875, valid accuracy 79.84%
GMM: epoch 10: train loss 662343.4375, valid accuracy 79.83%
GMM: epoch 11: train loss 662335.0625, valid accuracy 79.83%
GMM: epoch 12: train loss 662325.5, valid accuracy 79.85%
GMM: epoch 13: train loss 662316.6875, valid accuracy 79.87%
GMM: epoch 14: train loss 662307.0625, valid accuracy 79.88%
GMM: epoch 15: train loss 662298.0625, valid accuracy 79.90%
GMM: end: test accuracy 80.38%
我想通过评估精度对参数配置进行排序grep
,但出现以下错误
$ grep 'GMM: epoch 15: ' gmm_*ufactor100*.txt | sort -k 9 -n -r | head
exec: Failed to execute process '/usr/bin/grep': the total size of the argument list and exported variables (2.1MB) exceeds the OS limit of 1MB.
如何允许更大尺寸的参数列表?我使用的是 macOS Monterey,以防相关。