将文件夹拆分为子文件夹，保留父目录并基于百分比。使用终端

Question 1

这外壳检查-clean Bash 程序可能适合您的需要：

#! /bin/bash -p

#### Utility functions

function quit
{
    local -r msg=${1-unknown error}

    printf '%s: ERROR: %s\n' "$0" "$msg" >&2
    exit 1
}

# Do a random shuffle of the 'dirs' array
# (Code adapted from "BashFAQ/026 - Greg's Wiki",
# <https://mywiki.wooledge.org/BashFAQ/026>)
function shuffle_dirs
{
   local i tmp size max rand

   size=${#dirs[@]}
   for ((i=size-1; i>0; i--)); do
      # RANDOM % (i+1) is biased because of the limited range of $RANDOM
      # Compensate by using a range which is a multiple of the rand modulus.

      max=$(( 32768 / (i+1) * (i+1) ))
      while (( (rand=RANDOM) >= max )); do :; done
      rand=$(( rand % (i+1) ))
      tmp=${dirs[i]} dirs[i]=${dirs[rand]} dirs[rand]=$tmp
   done

   return 0
}

# Copy an FA/..._L/... directory and directories paired with it under $srcdir
# to a "split" directory ('test', 'train', or 'validation') under $destdir
function copy_dirs
{
    local -r splitdir=$1    # 'test', 'train', or 'validation'
    local -r fa_l_dir=$2    # E.g. 'FA/CN/CN_L/GE_FSPGR/b'

    local -a parts
    local p0 p2 path

    IFS=/ read -r -a parts <<<"$fa_l_dir"
    for p0 in FA T1; do
        for p2 in "${parts[2]}" "${parts[2]%_L}_R"; do
            path="$p0/${parts[1]}/$p2/${parts[3]}/${parts[4]}"
            srcpath="$srcdir/$path"

            destpath="$destdir/$splitdir/$path"
            mkdir --parents --verbose -- "${destpath%/*}"
            cp --archive --verbose -- "$srcpath" "$destpath"
        done
    done

    return 0
}

#### "main"

if (( $# != 2 )); then
    printf 'usage: %s SRCDIR DESTDIR\n' "$0" >&2
    exit 1
fi

declare -r srcdir=$1
declare -r destdir=$2

# Check for existence of required directories
for d in "$srcdir" "$srcdir/FA" "$srcdir/T1" "$destdir"; do
    [[ -e $d ]] || quit "'$d' does not exist"
    [[ -d $d ]] || quit "'$d' is not a directory"
done

shopt -s nullglob

for groupdir in "$srcdir"/FA/*/*_L/*/; do
    # Get the list of FA/..._L/group/... directories to be copied
    dirs=()
    for d in "$groupdir"*; do
        dirs+=( "${d#"$srcdir/"}" )
    done

    ndirs=${#dirs[*]}
    if (( ndirs == 0 )); then
        printf "%s: WARNING: no directories match '$groupdir*'\\n" "$0" >&2
        continue
    fi

    # Randomly shuffle the list of directories to randomize which directories
    # get copied to which locations
    shuffle_dirs

    declare -i pc_sum=0 lo=0 hi
    for splitdir_pc in test:60 train:20 validation:20; do
        splitdir=${splitdir_pc%:*}
        pc=${splitdir_pc#*:}

        pc_sum+=$pc
        hi='(ndirs * pc_sum + 50)/100 - 1'

        for ((i=lo; i<=hi; i++)); do
            copy_dirs "$splitdir" "${dirs[i]}"
        done

        lo='hi+1'
    done
done

exit 0

该程序采用两个命令行参数：
1. FA包含和目录的目录的路径T1。
2. test包含、train和validation目录（以及将在其下复制的目录）的目录路径。
如果您有兴趣，我很乐意向代码添加文档或回答您的任何问题。

Answer

这外壳检查-clean Bash 程序可能适合您的需要：

#! /bin/bash -p

#### Utility functions

function quit
{
    local -r msg=${1-unknown error}

    printf '%s: ERROR: %s\n' "$0" "$msg" >&2
    exit 1
}

# Do a random shuffle of the 'dirs' array
# (Code adapted from "BashFAQ/026 - Greg's Wiki",
# <https://mywiki.wooledge.org/BashFAQ/026>)
function shuffle_dirs
{
   local i tmp size max rand

   size=${#dirs[@]}
   for ((i=size-1; i>0; i--)); do
      # RANDOM % (i+1) is biased because of the limited range of $RANDOM
      # Compensate by using a range which is a multiple of the rand modulus.

      max=$(( 32768 / (i+1) * (i+1) ))
      while (( (rand=RANDOM) >= max )); do :; done
      rand=$(( rand % (i+1) ))
      tmp=${dirs[i]} dirs[i]=${dirs[rand]} dirs[rand]=$tmp
   done

   return 0
}

# Copy an FA/..._L/... directory and directories paired with it under $srcdir
# to a "split" directory ('test', 'train', or 'validation') under $destdir
function copy_dirs
{
    local -r splitdir=$1    # 'test', 'train', or 'validation'
    local -r fa_l_dir=$2    # E.g. 'FA/CN/CN_L/GE_FSPGR/b'

    local -a parts
    local p0 p2 path

    IFS=/ read -r -a parts <<<"$fa_l_dir"
    for p0 in FA T1; do
        for p2 in "${parts[2]}" "${parts[2]%_L}_R"; do
            path="$p0/${parts[1]}/$p2/${parts[3]}/${parts[4]}"
            srcpath="$srcdir/$path"

            destpath="$destdir/$splitdir/$path"
            mkdir --parents --verbose -- "${destpath%/*}"
            cp --archive --verbose -- "$srcpath" "$destpath"
        done
    done

    return 0
}

#### "main"

if (( $# != 2 )); then
    printf 'usage: %s SRCDIR DESTDIR\n' "$0" >&2
    exit 1
fi

declare -r srcdir=$1
declare -r destdir=$2

# Check for existence of required directories
for d in "$srcdir" "$srcdir/FA" "$srcdir/T1" "$destdir"; do
    [[ -e $d ]] || quit "'$d' does not exist"
    [[ -d $d ]] || quit "'$d' is not a directory"
done

shopt -s nullglob

for groupdir in "$srcdir"/FA/*/*_L/*/; do
    # Get the list of FA/..._L/group/... directories to be copied
    dirs=()
    for d in "$groupdir"*; do
        dirs+=( "${d#"$srcdir/"}" )
    done

    ndirs=${#dirs[*]}
    if (( ndirs == 0 )); then
        printf "%s: WARNING: no directories match '$groupdir*'\\n" "$0" >&2
        continue
    fi

    # Randomly shuffle the list of directories to randomize which directories
    # get copied to which locations
    shuffle_dirs

    declare -i pc_sum=0 lo=0 hi
    for splitdir_pc in test:60 train:20 validation:20; do
        splitdir=${splitdir_pc%:*}
        pc=${splitdir_pc#*:}

        pc_sum+=$pc
        hi='(ndirs * pc_sum + 50)/100 - 1'

        for ((i=lo; i<=hi; i++)); do
            copy_dirs "$splitdir" "${dirs[i]}"
        done

        lo='hi+1'
    done
done

exit 0

该程序采用两个命令行参数：
1. FA包含和目录的目录的路径T1。
2. test包含、train和validation目录（以及将在其下复制的目录）的目录路径。
如果您有兴趣，我很乐意向代码添加文档或回答您的任何问题。

Question 2

计算路径名或文件内容的哈希值。这是一个哈希函数示例。

$ date | shasum
7ae62fd4e6483c966fd23d8eeafabc934226914d  -

取第一个字符：... shasum | cut -c1

创建十六个符号链接，大致对应于您所需的百分比。将它们指向您的三个现有目录。

ln -s /data/test     0
...
ln -s /data/test     9

ln -s /data/train    a
ln -s /data/train    b
ln -s /data/train    c

ln -s /data/validate d
ln -s /data/validate e
ln -s /data/validate f

现在很容易了。获取哈希的初始字符以及cp该目标目录的 src 文件。

您将获得每个类别中大约正确数量的文件。如果您更喜欢 1/256 而不是 1/16 的分辨率，请随意为每个哈希的前两个字符创建符号链接。

编辑

你说你有一堆文件，例如

FA/CN/CN_L/GE_FSPGR/a/one.csv
FA/CN/CN_L/GE_FSPGR/a/two.csv
...
FA/CN/CN_L/GE_FSPGR/a/one_hundred.csv

您希望为每个文件选择三个条件之一：训练、测试、验证。

让我们散列一个特别的路径名：

$ echo FA/CN/CN_L/GE_FSPGR/a/one.csv | shasum
16930ea7216d650735ea7c223a67371c5a5ed770  -
$
$ echo FA/CN/CN_L/GE_FSPGR/a/one.csv | shasum | cut -c1
1

对其他文件重复此操作会产生d和2。

已设置的几个符号链接将导致$ cp one.csv 1和$ cp one_hundred.csv 2 降落在“测试”区域，并将$ cp two.csv d 成为验证数据的一部分。

Answer