如何使用命令行中给定的正则表达式查找远程 http/https 目录的大小(非递归)

如何使用命令行中给定的正则表达式查找远程 http/https 目录的大小(非递归)

我正在尝试使用命令行中给定的 regexp () 查找远程 http/https 目录的大小(非递归)\.mp4,是否有工具可以执行此操作,或者我是否需要解析该目录index.html

EDIT0:这是我想要解析的输出:

$ URL=https://cdn.sermons.love/mp4/Joseph%20Prince/
$ curl -s $URL | html2text -width $COLUMNS
****** Index of /mp4/Joseph Prince ******
[Icon ] Name                                                                           Last_modified      Size
Description
=======================================================================================================================
[[PARENTDIR]] Parent_Directory                                                                                    -
[[   ]] Joseph_Prince_-_Activate_the_Grace_Covenant_Through_Tongues.mp4                2019-04-16 22:32  428M
[[   ]] Joseph_Prince_-_Align_Yourself_With_His_Purpose_and_Prosper.mp4                2019-04-16 22:36  452M
[[   ]] Joseph_Prince_-_Amazing_Things_Happen_When_You_Flow_with_The_Spirit.mp4        2019-04-16 21:48  391M
[[   ]] Joseph_Prince_-_Are_You_Frustrating_The_Favor_of_God.mp4                       2019-04-16 21:27  524M
[[   ]] Joseph_Prince_-_As_Jesus_Is,_So_Are_You.mp4                                    2019-04-16 22:28  894M
[[   ]] Joseph_Prince_-_Blessings_Flow_Through_Grace.mp4                               2019-04-30 16:00  761M
[[   ]] Joseph_Prince_-_Break_Every_Bad_Habit_With_Christ.mp4                          2019-04-16 22:48  462M
[[   ]] Joseph_Prince_-_Break_Free_from_Addiction_and_Shame.mp4                        2019-04-16 22:12  388M
[[   ]] Joseph_Prince_-_Come_As_You_Are_and_Receive_Your_Miracle.mp4                   2019-04-16 22:16  444M
[[   ]] Joseph_Prince_-_Discerning_the_Lord's_Body_for_Greater_Health.mp4              2019-04-16 22:38  705M
[[   ]] Joseph_Prince_-_Discover_God's_Way_to_Bless_You.mp4                            2019-04-16 21:28  524M
[[   ]] Joseph_Prince_-_Don't_Fight_-_Feed!.mp4                                        2019-04-16 22:18  453M
[[   ]] Joseph_Prince_-_Draw_The_Blood_Line_Of_Protection.mp4                          2019-04-25 13:21  804M
[[   ]] Joseph_Prince_-_Easter_from_New_Creation_Church.mp4                            2019-04-16 21:23  843M
[[   ]] Joseph_Prince_-_Enjoy_Jesus'_Supply_And_Delight_His_Heart.mp4                  2019-04-16 21:38  609M
[[   ]] Joseph_Prince_-_Experience_God's_Restoration_For_Every_Regret.mp4              2019-04-16 22:37  421M
[[   ]] Joseph_Prince_-_Experience_The_Grace_Revolution.mp4                            2019-04-16 21:41  189M
[[   ]] Joseph_Prince_-_Find_Freedom_from_Every_Bondage_and_Addiction.mp4              2019-04-16 22:13  523M
[[   ]] Joseph_Prince_-_Find_True_Fulfillment_In_Life.mp4                              2019-04-16 22:20  209M
[[   ]] Joseph_Prince_-_Five_Words_To_Live_By__The_Battle_Is_The_Lord's.mp4            2019-04-16 21:57  226M
[[   ]] Joseph_Prince_-_Fresh_Grace_For_Every_Trial.mp4                                2019-04-16 22:14  609M
[[   ]] Joseph_Prince_-_Give_Jesus_Your_Cares_And_Live_Stress-Free.mp4                 2019-04-16 21:00  428M
[[   ]] Joseph_Prince_-_God_Is_A_Gracious_Rewarder.mp4                                 2019-04-16 22:42  224M
[[   ]] Joseph_Prince_-_God's_Blueprint_For_Leadership.mp4                             2019-04-16 21:30  776M
[[   ]] Joseph_Prince_-_God's_Perfect_Timing_In_The_Christmas_Story.mp4                2019-04-16 21:29  441M
[[   ]] Joseph_Prince_-_God's_Plan_To_Bless_You.mp4                                    2019-04-16 21:56  200M
[[   ]] Joseph_Prince_-_Godly_Discipline_And_Correction_Brings_Promotion.mp4           2019-04-16 21:56  776M
[[   ]] Joseph_Prince_-_Got_A_Weakness__God_Can_Use_You!.mp4                           2019-04-16 21:25  524M
[[   ]] Joseph_Prince_-_Grace_Leadership_In_Action__How_To_Represent_God's_Heart.mp4   2019-04-16 22:30  732M
[[   ]] Joseph_Prince_-_He_Is_Risen.mp4                                                2019-04-16 21:47  442M
[[   ]] Joseph_Prince_-_Healing_Flows_When_Grace_Is_Exalted.mp4                        2019-04-16 21:51  225M
[[   ]] Joseph_Prince_-_Hear_Jesus_Only_And_Be_Uplifted.mp4                            2019-04-16 21:25  436M
[[   ]] Joseph_Prince_-_Hear_the_Preached_Word_and_See_Breakthroughs.mp4               2019-04-16 21:31  459M
[[   ]] Joseph_Prince_-_Hesed_Wisdom_to_Live_Skillfully.mp4                            2019-04-16 21:32  389M
[[   ]] Joseph_Prince_-_His_Healing_Is_For_The_Undeserving.mp4                         2019-04-16 22:20  217M
[[   ]] Joseph_Prince_-_His_Promises_Are_Yours_To_Own.mp4                              2019-04-16 22:23  524M
[[   ]] Joseph_Prince_-_His_Radiance_Upon_You_Brings_Favor.mp4                         2019-04-16 22:00  689M
[[   ]] Joseph_Prince_-_His_Resurrection,_Proof_Of_Your_Righteousness.mp4              2019-04-19 11:03  809M
[[   ]] Joseph_Prince_-_How_To_Be_Blessed_God's_Way.mp4                                2019-04-16 21:08  776M
[[   ]] Joseph_Prince_-_How_To_Live_Free_From_The_Curse.mp4                            2019-04-16 22:20  614M
[[   ]] Joseph_Prince_-_How_To_Make_Spirit-Led_Decisions.mp4                           2019-04-16 21:14  871M
[[   ]] Joseph_Prince_-_How_To_Pray_When_You_Have_No_Prayer.mp4                        2019-04-16 22:31  372M
[[   ]] Joseph_Prince_-_How_You_See_Jesus_Is_How_You_Will_Receive.mp4                  2019-04-16 21:50  657M
[[   ]] Joseph_Prince_-_Immanuel__What_It_Means_To_Have_The_Lord_With_You.mp4          2019-04-16 21:30  210M
[[   ]] Joseph_Prince_-_Inherit_God's_Promises_By_Faith,_Not_by_Works.mp4              2019-04-16 21:49  521M
[[   ]] Joseph_Prince_-_Jesus_Draws_Near_When_You_Are_Discouraged_(Live_In_Israel).mp4 2019-04-16 21:18  561M
[[   ]] Joseph_Prince_-_Jesus_Our_Jubilee.mp4                                          2019-04-16 21:53  694M
[[   ]] Joseph_Prince_-_Jesus__Your_Reason_For_A_Fear-Free_Life.mp4                    2019-04-16 21:05  210M
[[   ]] Joseph_Prince_-_Keys_To_Healing_In_The_Hebrew_Language.mp4                     2019-04-16 21:00  250M
[[   ]] Joseph_Prince_-_Last_To_First_When_You_Trust_His_Goodness.mp4                  2019-04-16 22:50  781M
[[   ]] Joseph_Prince_-_Learn_to_See_What_God_Sees.mp4                                 2019-04-16 21:46  419M
[[   ]] Joseph_Prince_-_Let_Go_And_Let_His_Supply_Flow.mp4                             2019-04-16 22:33  461M
[[   ]] Joseph_Prince_-_Let_Go_and_Flow_in_the_Vine_Life.mp4                           2019-04-16 22:26  458M
[[   ]] Joseph_Prince_-_Live_Bold_Without_Guilt_and_Fear.mp4                           2019-04-16 22:04  427M
[[   ]] Joseph_Prince_-_Live_Confident.mp4                                             2019-04-16 21:03  426M
[[   ]] Joseph_Prince_-_Live_Life_Loved_By_The_Shepherd.mp4                            2019-04-16 21:39  428M
[[   ]] Joseph_Prince_-_Live_Long,_Live_Strong.mp4                                     2019-04-16 22:35  621M
[[   ]] Joseph_Prince_-_Live_Strong_In_The_Father's_Love.mp4                           2019-04-16 21:11  465M
[[   ]] Joseph_Prince_-_Live_Undefeated_In_Christ.mp4                                  2019-04-16 21:57  438M
[[   ]] Joseph_Prince_-_Make_Grace_Your_Way_of_Life.mp4                                2019-04-16 22:15  520M
[[   ]] Joseph_Prince_-_Move_From_Predicament_To_Promotion.mp4                         2019-04-16 22:46  813M
[[   ]] Joseph_Prince_-_No_Condemnation_Leads_to_Divine_Health.mp4                     2019-04-16 21:59  685M
[[   ]] Joseph_Prince_-_Not_Ashamed_Of_The_Gospel.mp4                                  2019-04-16 22:22  235M
[[   ]] Joseph_Prince_-_Practical_Leadership_Keys_To_Living_Holy.mp4                   2019-04-16 21:36  829M
[[   ]] Joseph_Prince_-_Receive_God's_Supply_for_All_of_Life's_Demands.mp4             2019-04-16 22:44  525M
[[   ]] Joseph_Prince_-_Redemption_Truths_That_Bless_Your_Relationships.mp4            2019-04-16 21:09  217M
[[   ]] Joseph_Prince_-_Rest_And_Receive_At_Jesus'_Feet.mp4                            2019-04-16 21:54  449M
[[   ]] Joseph_Prince_-_Rest_In_Jesus'_Faith_For_Miracles.mp4                          2019-04-16 21:04  519M
[[   ]] Joseph_Prince_-_Rest_Till_Your_Enemies_Become_Your_Footstool.mp4               2019-04-16 20:59  472M
[[   ]] Joseph_Prince_-_Rest!_God_Is_Working_Behind_The_Scenes.mp4                     2019-04-16 22:43  523M
[[   ]] Joseph_Prince_-_Say_Amen_To_God's_Promises.mp4                                 2019-04-16 21:45  521M
[[   ]] Joseph_Prince_-_Set_Apart_To_Be_Kings_And_Priests.mp4                          2019-04-16 21:34  802M
[[   ]] Joseph_Prince_-_Set_Free_to_Reign_in_Life.mp4                                  2019-04-16 22:06  528M
[[   ]] Joseph_Prince_-_Speak_Out_and_Find_Strength.mp4                                2019-04-16 22:11  1.1G
[[   ]] Joseph_Prince_-_Speak_Out_by_Faith_and_Win.mp4                                 2019-04-16 21:20  412M
[[   ]] Joseph_Prince_-_Stay_on_Grace_Ground_and_Experience_True_Life.mp4              2019-04-16 21:41  1.0G
[[   ]] Joseph_Prince_-_Stronger_Through_Every_Trial_And_Battle.mp4                    2019-04-16 22:47  789M
[[   ]] Joseph_Prince_-_The_Four_Gospels_Unlocked_for_Your_Blessings.mp4               2019-04-16 22:17  922M
[[   ]] Joseph_Prince_-_The_Friend_You_Can_Always_Depend_On.mp4                        2019-04-16 22:40  1.1G
[[   ]] Joseph_Prince_-_The_God_Who_Goes_Before.mp4                                    2019-04-16 22:41  442M
[[   ]] Joseph_Prince_-_The_Health-Giving_Power_Of_A_Relaxed_Heart.mp4                 2019-04-16 21:10  613M
[[   ]] Joseph_Prince_-_The_Heart_of_the_Father_Revealed.mp4                           2019-04-16 21:44  468M
[[   ]] Joseph_Prince_-_The_Lord_Our_Righteousness.mp4                                 2019-04-16 21:02  423M
[[   ]] Joseph_Prince_-_The_Secret_of_Hearing_That_Brings_Untold_Blessings.mp4         2019-04-16 22:25  687M
[[   ]] Joseph_Prince_-_The_Spirit's_Rivers_Of_Provision_And_Healing.mp4               2019-04-16 21:52  404M
[[   ]] Joseph_Prince_-_Turn_Your_Frustrations_Into_Breakthroughs.mp4                  2019-04-16 21:12  461M
[[   ]] Joseph_Prince_-_Unlocking_Redemption's_Blessings_In_Your_Life.mp4              2019-04-16 21:43  686M
[[   ]] Joseph_Prince_-_Victory_in_Your_Day_of_Trouble.mp4                             2019-04-16 22:05  456M
[[   ]] Joseph_Prince_-_Walk_In_Constant_Victory_Over_Fear.mp4                         2019-04-16 22:09  1.1G
[[   ]] Joseph_Prince_-_What_Is_Earnest_Prayer_To_God.mp4                              2019-04-16 22:31  215M
[[   ]] Joseph_Prince_-_What_Makes_No_Weapon_Prosper_Against_You.mp4                   2019-04-30 15:31  253M
[[   ]] Joseph_Prince_-_Where_Is_God_In_The_Midst_Of_Your_Trouble.mp4                  2019-04-16 21:17  833M
[[   ]] Joseph_Prince_-_Win_Over_Discouragement,_Depression_and_Burnout.mp4            2019-04-16 22:24  396M
[[   ]] Joseph_Prince_-_Win_Over_Fear_and_Pride.mp4                                    2019-04-16 21:19  459M
[[   ]] Joseph_Prince_-_Win_Over_Guilt_and_Condemnation.mp4                            2019-04-16 21:15  223M
[[   ]] Joseph_Prince_-_Wisdom_For_Holy_Living.mp4                                     2019-04-16 21:07  791M
[[   ]] Joseph_Prince_-_You_Are_Forgiven.mp4                                           2019-04-16 22:04  440M
[[   ]] Joseph_Prince_-_You_Can_Live_Healed.mp4                                        2019-04-16 22:33  217M
[[   ]] Joseph_Prince_-_Your_Blessed_Hope_In_Dark_Times.mp4                            2019-04-16 22:21  464M
[[   ]] Joseph_Prince_-_Your_Only_Battle_is_Fight_to_Remain_at_Rest.mp4                2019-04-16 21:01  616M
[[   ]] Joseph_Prince_-_Your_Past_Does_Not_Determine_Your_Future.mp4                   2019-04-16 21:22  685M
[[   ]] Joseph_Prince_-_Your_Reason_For_A_Fear_Free_Life.mp4                           2019-04-16 21:13  468M
[[   ]] Joseph_Prince_-_Your_Security_in_Time_of_Shaking.mp4                           2019-04-16 22:03  681M
=======================================================================================================================

编辑1:我尝试了这一点,但它不起作用,因为没有添加演出中的文件大小:

$ curl -s $URL | html2text -width $COLUMNS | sed "s/M$//;s/G$/*1024/" | awk '/\.mp4/{size=$6;print"=> 6th field = "$6" size = "size >"/dev/stderr";total+=size}END{print "=> total = "total}'
=> 6th field = 428 size = 428
=> 6th field = 452 size = 452
=> 6th field = 391 size = 391
=> 6th field = 524 size = 524
=> 6th field = 894 size = 894
=> 6th field = 462 size = 462
=> 6th field = 388 size = 388
=> 6th field = 444 size = 444
=> 6th field = 705 size = 705
=> 6th field = 524 size = 524
=> 6th field = 453 size = 453
=> 6th field = 843 size = 843
=> 6th field = 609 size = 609
=> 6th field = 421 size = 421
=> 6th field = 189 size = 189
=> 6th field = 523 size = 523
=> 6th field = 209 size = 209
=> 6th field = 226 size = 226
=> 6th field = 609 size = 609
=> 6th field = 428 size = 428
=> 6th field = 224 size = 224
=> 6th field = 776 size = 776
=> 6th field = 441 size = 441
=> 6th field = 200 size = 200
=> 6th field = 776 size = 776
=> 6th field = 524 size = 524
=> 6th field = 732 size = 732
=> 6th field = 442 size = 442
=> 6th field = 225 size = 225
=> 6th field = 436 size = 436
=> 6th field = 459 size = 459
=> 6th field = 389 size = 389
=> 6th field = 217 size = 217
=> 6th field = 524 size = 524
=> 6th field = 689 size = 689
=> 6th field = 809 size = 809
=> 6th field = 776 size = 776
=> 6th field = 614 size = 614
=> 6th field = 871 size = 871
=> 6th field = 372 size = 372
=> 6th field = 657 size = 657
=> 6th field = 210 size = 210
=> 6th field = 521 size = 521
=> 6th field = 561 size = 561
=> 6th field = 694 size = 694
=> 6th field = 210 size = 210
=> 6th field = 250 size = 250
=> 6th field = 781 size = 781
=> 6th field = 419 size = 419
=> 6th field = 461 size = 461
=> 6th field = 458 size = 458
=> 6th field = 427 size = 427
=> 6th field = 426 size = 426
=> 6th field = 428 size = 428
=> 6th field = 621 size = 621
=> 6th field = 465 size = 465
=> 6th field = 438 size = 438
=> 6th field = 520 size = 520
=> 6th field = 813 size = 813
=> 6th field = 685 size = 685
=> 6th field = 235 size = 235
=> 6th field = 829 size = 829
=> 6th field = 525 size = 525
=> 6th field = 217 size = 217
=> 6th field = 449 size = 449
=> 6th field = 519 size = 519
=> 6th field = 472 size = 472
=> 6th field = 523 size = 523
=> 6th field = 521 size = 521
=> 6th field = 802 size = 802
=> 6th field = 528 size = 528
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 412 size = 412
=> 6th field = 1.0*1024 size = 1.0*1024
=> 6th field = 789 size = 789
=> 6th field = 922 size = 922
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 442 size = 442
=> 6th field = 613 size = 613
=> 6th field = 468 size = 468
=> 6th field = 423 size = 423
=> 6th field = 687 size = 687
=> 6th field = 404 size = 404
=> 6th field = 461 size = 461
=> 6th field = 686 size = 686
=> 6th field = 456 size = 456
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 215 size = 215
=> 6th field = 450 size = 450
=> 6th field = 833 size = 833
=> 6th field = 396 size = 396
=> 6th field = 459 size = 459
=> 6th field = 223 size = 223
=> 6th field = 791 size = 791
=> 6th field = 440 size = 440
=> 6th field = 217 size = 217
=> 6th field = 464 size = 464
=> 6th field = 616 size = 616
=> 6th field = 685 size = 685
=> 6th field = 468 size = 468
=> 6th field = 681 size = 681
=> total = 49588.3

EDIT2:我终于在 Perl 中做到了:

$ curl -s $URL | html2text -width $COLUMNS | perl -n -ale 'if(/mp4 /){$F[-1] =~ s/M$//;$F[-1] =~ s/G$/*1024/;$total+=eval $F[-1]}END{print "=> total = ",$total}'
=> total = 53987.2

pupEDIT3:我也尝试了and的组合,jq但日期和大小似乎不在数组中,因此不容易解析:

$ curl -s $URL | pup 'json{}' | jq '.[]  | .children[0].children[1].children[1].text' 
"- 2019-04-16 22:32  428M 2019-04-16 22:36  452M 2019-04-16 21:48  391M 2019-04-16 21:27  524M 2019-04-16 22:28  894M 2019-04-30 16:00  761M 2019-04-16 22:48  462M 2019-04-16 22:12  388M 2019-04-16 22:16  444M 2019-04-16 22:38  705M 2019-04-16 21:28  524M 2019-04-16 22:18  453M 2019-04-25 13:21  804M 2019-04-16 21:23  843M 2019-04-16 21:38  609M 2019-04-16 22:37  421M 2019-04-16 21:41  189M 2019-04-16 22:13  523M 2019-04-16 22:20  209M 2019-04-16 21:57  226M 2019-04-16 22:14  609M 2019-04-16 21:00  428M 2019-04-16 22:42  224M 2019-04-16 21:30  776M 2019-04-16 21:29  441M 2019-04-16 21:56  200M 2019-04-16 21:56  776M 2019-04-16 21:25  524M 2019-04-16 22:30  732M 2019-04-16 21:47  442M 2019-04-16 21:51  225M 2019-04-16 21:25  436M 2019-04-16 21:31  459M 2019-04-16 21:32  389M 2019-04-16 22:20  217M 2019-04-16 22:23  524M 2019-04-16 22:00  689M 2019-04-19 11:03  809M 2019-04-16 21:08  776M 2019-04-16 22:20  614M 2019-04-16 21:14  871M 2019-04-16 22:31  372M 2019-04-16 21:50  657M 2019-04-16 21:30  210M 2019-04-16 21:49  521M 2019-04-16 21:18  561M 2019-04-16 21:53  694M 2019-04-16 21:05  210M 2019-04-16 21:00  250M 2019-04-16 22:50  781M 2019-04-16 21:46  419M 2019-04-16 22:33  461M 2019-04-16 22:26  458M 2019-04-16 22:04  427M 2019-04-16 21:03  426M 2019-04-16 21:39  428M 2019-04-16 22:35  621M 2019-04-16 21:11  465M 2019-04-16 21:57  438M 2019-04-16 22:15  520M 2019-04-16 22:46  813M 2019-04-16 21:59  685M 2019-04-16 22:22  235M 2019-04-16 21:36  829M 2019-04-16 22:44  525M 2019-04-16 21:09  217M 2019-04-16 21:54  449M 2019-04-16 21:04  519M 2019-04-16 20:59  472M 2019-04-16 22:43  523M 2019-04-16 21:45  521M 2019-04-16 21:34  802M 2019-04-16 22:06  528M 2019-04-16 22:11  1.1G 2019-04-16 21:20  412M 2019-04-16 21:41  1.0G 2019-04-16 22:47  789M 2019-04-16 22:17  922M 2019-04-16 22:40  1.1G 2019-04-16 22:41  442M 2019-04-16 21:10  613M 2019-04-16 21:44  468M 2019-04-16 21:02  423M 2019-04-16 22:25  687M 2019-04-16 21:52  404M 2019-04-16 21:12  461M 2019-04-16 21:43  686M 2019-04-16 22:05  456M 2019-04-16 22:09  1.1G 2019-04-16 22:31  215M 2019-04-30 15:31  253M 2019-04-16 21:17  833M 2019-04-16 22:24  396M 2019-04-16 21:19  459M 2019-04-16 21:15  223M 2019-04-16 21:07  791M 2019-04-16 22:04  440M 2019-04-16 22:33  217M 2019-04-16 22:21  464M 2019-04-16 21:01  616M 2019-04-16 21:22  685M 2019-04-16 21:13  468M 2019-04-16 22:03  681M"

难道没有更简单或“更干净”的方法吗?

答案1

使用numfmt --from=...将文件大小转换为字节,然后您可以轻松地总结它们:

curl -s $URL | html2text -width $COLUMNS \
| awk '$3~/\.mp4/{print $6}' \
| LC_ALL=C numfmt --from=iec \
| paste -sd+ | bc

58044134197
  • 添加| numfmt --to=iec以再次获得人类可读的大小,如果您想--format '%.1f'获得小数位,则添加。
  • 用于--from=si基于 1000 的文件大小。
  • 根据您的本地设置,您可以省略LC_ALL=C,但我收到错误1.1G

答案2

尝试这个,

cat file | awk '{($NF ~ /G/)?$NF=$NF*1024:$NF=$NF*1} {sum += $NF} END {print sum}'
  • 如果最后一个字段有,则将其乘以 1024G
  • 否则乘以1(只是为了删除M,如果你有K或其他东西,应该小心)
  • 最后,最后一个字段中所有值的总和。

也许你可以用 6 替换 NF,因为你提到它是第 6 个字段而不是最后一个字段

相关内容