如何从代币列表中删除材料？

Question

这是如何将材料附加到令牌列表？

宏\prependto和\appendto工作很好添加内容到标记列表。如何创建类似 和 的宏\gobblefirst来\gobblelast从列表中删除第一个/最后一个标记？

假设以下情况：

1. 这些标记由一些固定的标记分隔（例如\relax）；或者
2. 这些代币是不是由任意固定标记分隔。

我认为 1 的解决方案比 2 的解决方案要容易得多。

Answer 1

当你想要维护可以在两端添加或删除某些内容的标记列表时，最好添加分隔符：

\catcode`@=11 % \makeatletter
\def\prependtolist#1#2{% #1=list name, #2=item to add
  \toks@={\listitem{#2}}%
  \toks@=\expandafter{\the\expandafter\toks@#1}%
  \edef#1{\the\toks@}
}
\def\appendtolist#1#2{%
  \expandafter\def\expandafter#1\expandafter{#1\listitem{#2}}%
}
\def\removetop#1#2{% #1=list,#2=macro to store the removed item
  \expandafter\removetopaux#1\removetop{#1}{#2}}
\def\removetopaux\listitem#1#2\removetop#3#4{%
  \def#2{#1}%
  \def#3{#2}%
}

\catcode`@=12

（有更好的方法，这只是为了增添风味）。

当必须使用列表时，只需设置

\def\listitem#1{#1}

LaTeX3 提供了一系列工具来简化这项工作。“序列”数据类型是最简单的：

\usepackage{expl3}
\seq_new:N \l_werner_my_seq
\seq_put_left:Nn \l_werner_my_seq {a} % analog of \prependtolist
\seq_put_right:Nn \l_werner_my_seq {a} % analog of \appendtolist
\seq_pop_left:NN \l_werner_my_seq \l_tmpa_tl % analog of \removetop
\seq_pop_right:NN \l_werner_my_seq \l_tmpa_tl % analog of \removebottom 

% How to deliver the contents of a sequence
\seq_map_function:NN \l_werner_my_seq \use:n

最后一行显示了如何传递列表的内容（隐式分隔符被重新定义为不执行任何操作）。

使用标记列表也是可能的，但可以删除项目而不是标记，其中一个项目要么是既不是明确括号的标记，要么是带括号的组。

\tl_new:N \l_werner_my_tl
\tl_put_left:Nn \l_werner_my_tl {a} % analog of \prependto
\tl_put_right:Nn \l_werner_my_tl {a} % analog of \appendto
\tl_set:Nx \l_werner_my_tl { \tl_tail:N \l_werner_my_tl } % your \gobblefirst

\gobblelast通过两次反转标记列表可以获得宏：

\cs_generate_variant:Nn \tl_reverse:n {f}
\tl_set:Nx \l_werner_my_tl { \tl_reverse:f { \tl_tail:f { \tl_reverse:V \l_werner_my_tl } } }

“token列表”方法的实现：

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\uselist}{ m }{ \tl_use:c { l_werner_#1_tl } }
\NewDocumentCommand{\createlist}{ m }{ \tl_new:c { l_werner_#1_tl } }
\NewDocumentCommand{\prependtolist}{ m m }
 {
  \werner_prepend:nn { #1 } { #2 }
 }
\NewDocumentCommand{\appendtolist}{ m m }
 {
  \werner_append:nn { #1 } { #2 }
 }
\NewDocumentCommand{\gobblefirst}{ m }
 {
  \werner_gobble_first:n { #1 }
 }
\NewDocumentCommand{\gobblelast}{ m }
 {
  \werner_gobble_last:n { #1 }
 }

\cs_new_protected:Npn \werner_prepend:nn #1 #2
 {
  \tl_put_left:cn { l_werner_#1_tl } { #2 }
 }
\cs_new_protected:Npn \werner_append:nn #1 #2
 {
  \tl_put_right:cn { l_werner_#1_tl } { #2 }
 }
\cs_new_protected:Npn \werner_gobble_first:n #1
 {
  \tl_set:cx { l_werner_#1_tl } { \tl_tail:v { l_werner_#1_tl } }
 }
\cs_new_protected:Npn \werner_gobble_last:n #1
 {
  \tl_set:cx { l_werner_#1_tl } { \tl_reverse:v { l_werner_#1_tl } }
  \werner_gobble_first:n { #1 }
  \tl_set:cx { l_werner_#1_tl } { \tl_reverse:v { l_werner_#1_tl } }
 }
\cs_generate_variant:Nn \tl_reverse:n {v}
\NewDocumentCommand{\showlist}{m}{ \tl_show:c { l_werner_#1_tl } }

\ExplSyntaxOff

\createlist{my}
\appendtolist{my}{z}
\prependtolist{my}{{ab}c}
\appendtolist{my}{\texttt}

\showlist{my}    % {ab}cz\texttt
\gobblefirst{my}
\showlist{my}    % cz\texttt
\gobblelast{my}
\showlist{my}    % cz
\gobblelast{my}
\showlist{my}    % c

相反，\showlist您可以说\uselist{my}传递列表的内容。

实施“序列”方法

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn

\NewDocumentCommand{\createlist}{ m }{ \seq_new:c { l_werner_#1_seq } }
\NewDocumentCommand{\uselist}{ m } { \seq_map_function:cN { l_werner_#1_seq } \use:n }
\NewDocumentCommand{\showlist}{ m } { \seq_show:c { l_werner_#1_seq } }
\NewDocumentCommand{\prependtolist}{ m m }{ \werner_prepend:nn { #1 } { #2 } }
\NewDocumentCommand{\appendtolist}{ m m }{ \werner_append:nn { #1 } { #2 } }
\NewDocumentCommand{\gobblefirst}{ m }{ \werner_gobble_first:n { #1 } }
\NewDocumentCommand{\gobblelast}{ m }{ \werner_gobble_last:n { #1 } }

\cs_new_protected:Npn \werner_prepend:nn #1 #2
 {
  \seq_put_left:cn { l_werner_#1_seq } { #2 }
 }
\cs_new_protected:Npn \werner_append:nn #1 #2
 {
  \seq_put_right:cn { l_werner_#1_seq } { #2 }
 }
\cs_new_protected:Npn \werner_gobble_first:n #1
 {
  \seq_pop_left:cN { l_werner_#1_seq } \l_tmpa_tl
 }
\cs_new_protected:Npn \werner_gobble_last:n #1
 {
  \seq_pop_right:cN { l_werner_#1_seq } \l_tmpa_tl
 }

\ExplSyntaxOff

输入与之前相同，不同之处在于\gobblefirst会删除{ab}c而不是{ab}。

Answer 2

我认为您需要对令牌列表中允许的内容给出一些条件。

令牌寄存器可以包含任意平衡列表，例如{ab}{cd}8 个令牌，但当您说“删除第一个令牌”时，您的意思是获得一个列表{cd}，就好像您只是删除了一样，{您得到的列表ab}{cd}不能存储在令牌寄存器中。（尽管如果您真的想要，您可以将其转换为ab\egroup{cd}在某些方面类似并且可以存储的）。但是我假设您想将括号组视为令牌。其他有趣的令牌是#和不匹配\if等。

因此，从一开始就可以进行删除，基本上没有任何限制：

\newtoks\aa

\def\gobble#1{}

\def\gobblefirst{%
  \aa\expandafter\expandafter\expandafter{\expandafter\gobble\the\aa}}


\aa{a\foo#{this and that}\hhh \if \egroup  \bgroup}\showthe\aa

\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa
\gobblefirst\showthe\aa


\bye

生成（如果通过纯 TeX 运行）

> a\foo ##{this and that}\hhh \if \egroup \bgroup .
l.10 ...that}\hhh \if \egroup  \bgroup}\showthe\aa

? 
> \foo ##{this and that}\hhh \if \egroup \bgroup .
l.12 \gobblefirst\showthe\aa

? 
> ##{this and that}\hhh \if \egroup \bgroup .
l.13 \gobblefirst\showthe\aa

? 
> {this and that}\hhh \if \egroup \bgroup .
l.14 \gobblefirst\showthe\aa

? 
> \hhh \if \egroup \bgroup .
l.15 \gobblefirst\showthe\aa

? 
> \if \egroup \bgroup .
l.16 \gobblefirst\showthe\aa

? 
> \egroup \bgroup .
l.17 \gobblefirst\showthe\aa

? 
> \bgroup .
l.18 \gobblefirst\showthe\aa

? 
> .
l.19 \gobblefirst\showthe\aa

我看不出有任何方法可以从另一端以相同的普遍性中删除，您总是必须施加一些限制，例如可能不会出现在列表中的保留“标记”标记，或者没有大括号组或......

最后吞噬

这个版本有一些限制，最主要的是令牌\xlast可能不在列表中，其他限制我留给毫无戒心的用户作为陷阱......

\def\xlast#1{%
\ifx\xlast#1%
 \else
 \bb\expandafter{\the\expandafter\bb\the\cc}%
 \cc{#1}%
 \expandafter\ifx\expandafter\xlast\gobble#1\xlast
 \else
 \expandafter\cc\expandafter{\expandafter{\the\cc}}%
 \fi
 \expandafter\xlast
\fi}

\newtoks\bb
\newtoks\cc
\def\gobblelasta{%
\bb{}%
\cc{}%
\expandafter\xlast\the\aa\xlast
\aa\bb}

\aa{a\foo#{this and that}\hhh  \fi \egroup  \bgroup}\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa
\gobblelasta\showthe\aa

\bye

从纯 TeX 生成以下内容：

> a\foo ##{this and that}\hhh \fi \egroup \bgroup .
l.46 ...hat}\hhh  \fi \egroup  \bgroup}\showthe\aa

? 
> a\foo ##{this and that}\hhh \fi \egroup .
l.47 \gobblelasta\showthe\aa

? 
> a\foo ##{this and that}\hhh \fi .
l.48 \gobblelasta\showthe\aa

? 
> a\foo ##{this and that}\hhh .
l.49 \gobblelasta\showthe\aa

? 
> a\foo ##{this and that}.
l.50 \gobblelasta\showthe\aa

? 
> a\foo ##.
l.51 \gobblelasta\showthe\aa

? 
> a\foo .
l.52 \gobblelasta\showthe\aa

? 
> a.
l.53 \gobblelasta\showthe\aa

? 
> .
l.54 \gobblelasta\showthe\aa

Answer 3

以下有三个版本\gobblelast：

\makeatletter

\def\gobblelast@nil{gobblelast@nil}

\newtoks\gobblelast@toks

\let\gobblelast@obr={
\let\gobblelast@cbr=}
\def\gobblelast@mkspc#1%
{%
  \expandafter\futurelet\expandafter\gobblelast@spc\expandafter\@gobble
}
\futurelet\next\gobblelast@mkspc a b

\newcommand\gobblelast[2]
{%
  \global\@tempcnta#2\relax
  \global\advance\@tempcnta\@ne
  \global\gobblelast@toks={}%
  \begingroup
  \let\@gobblelast@iter\@@gobblelast@iter
  \expandafter\futurelet\expandafter\next\expandafter\gobblelast@iter
  \the#1\relax\gobblelast@nil
  #1=\expandafter{\the\gobblelast@toks}%
}

\newcommand\gobblelast@iter
{%
  \gobblelast@ifx{\next\gobblelast@nil}
  {\endgroup\@gobble}
  {%
    \gobblelast@ifx{\next\gobblelast@obr}
    {\gobblelast@readobr}
    {%
      \gobblelast@ifx{\next\gobblelast@cbr}
      {\gobblelast@readcbr}
      {%
        \gobblelast@ifx{\next\gobblelast@spc}
        {\gobblelast@readspc}
        {\gobblelast@readtoken}
      }%
    }%
  }%
}

\newcommand\gobblelast@ifx[1]
{%
  \ifx#1\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi
}

\newcommand\gobblelast@readobr
{%
  \afterassignment\@gobblelast@readobr\let\next=%
}%

\newcommand\@gobblelast@readobr
{%
  \toks0\expandafter\expandafter\expandafter{\expandafter\string\expandafter{\iffalse}\fi}%
  \@gobblelast@iter
}

\newcommand\gobblelast@readcbr
{%
  \afterassignment\@gobblelast@readcbr\let\next=%
}%

\newcommand\@gobblelast@readcbr
{%
  \toks0\expandafter\expandafter\expandafter{\expandafter\iffalse\expandafter{\expandafter\fi\string}}%
  \@gobblelast@iter
}

\def\gobblelast@readspc%
{\afterassignment\@gobblelast@readspc\@tempcntb=`}

\def\@gobblelast@readspc%
{%
  \toks0{ }%
  \@gobblelast@iter
}

\def\gobblelast@readtoken#1%
{%
  \toks0{#1}%
  \@gobblelast@iter
}

\newcommand\@@gobblelast@iter
{%
  \begingroup
    \aftergroup\gobblelast@finish
    \futurelet\next\gobblelast@iter    
}

\newcommand\gobblelast@finish
{%
  \ifnum\@tempcnta>\z@
    \global\advance\@tempcnta\m@ne
   \else
    \global\gobblelast@toks=\expandafter{\the\toks\expandafter\z@\the\gobblelast@toks}%
  \fi
  \endgroup
}

\newcommand\weakgobblelast[2]
{%
  \global\@tempcnta#2\relax
  \global\advance\@tempcnta\@ne
  \global\gobblelast@toks={}%
  \begingroup
  \let\@gobblelast@iter\@weak@gobblelast@iter
  \expandafter\futurelet\expandafter\next\expandafter\weakgobblelast@iter
  \the#1\relax\gobblelast@nil
  #1=\expandafter{\the\gobblelast@toks}%
}

\newcommand\weakgobblelast@iter
{%
  \gobblelast@ifx{\next\gobblelast@nil}
  {\endgroup\@gobble}
  {%
    \gobblelast@ifx{\next\gobblelast@obr}
    {\gobblelast@readbr}
    {%
      \gobblelast@ifx{\next\gobblelast@spc}
      {\gobblelast@readspc}
      {\gobblelast@readtoken}
    }%
  }%
}

\newcommand\@weak@gobblelast@iter
{%
  \begingroup
    \aftergroup\gobblelast@finish
    \futurelet\next\weakgobblelast@iter    
}

\newcommand\gobblelast@readbr[1]
{%
  \toks0{{#1}}%
  \@gobblelast@iter
}%

\newcommand\voodoogobblelast[2]
{%
  \global\@tempcnta#2\relax
  \global\advance\@tempcnta\@ne
  \global\gobblelast@toks={}%
  \global\gobblelast@temptoks{}
  \begingroup
  \let\@gobblelast@iter\@voodoo@gobblelast@iter
  \expandafter\futurelet\expandafter\next\expandafter\voodoogobblelast@iter
  \the#1\relax\gobblelast@nil
  \edef\tmp{\expandafter\unexpanded\expandafter{\expandafter\unexpanded\expandafter{\the\gobblelast@temptoks}}\the\gobblelast@toks}
  #1=\expandafter{\tmp}%
}

\newcommand\voodoogobblelast@iter
{%
  \gobblelast@ifx{\next\gobblelast@nil}
  {\endgroup\@gobble}
  {%
    \gobblelast@ifx{\next\gobblelast@obr}
    {\vgobblelast@readobr}
    {%
      \gobblelast@ifx{\next\gobblelast@cbr}
      {\vgobblelast@readcbr}
      {%
        \gobblelast@ifx{\next\gobblelast@spc}
        {\gobblelast@readspc}
        {\gobblelast@readtoken}
      }%
    }%
  }%
}

\newcommand\@voodoo@gobblelast@iter
{%
  \begingroup
    \aftergroup\voodoogobblelast@finish
    \futurelet\next\voodoogobblelast@iter    
}


\newcommand\vgobblelast@readobr
{%
  \afterassignment\v@gobblelast@readobr\let\next=%
}%

\newcommand\gobblelast@vobr{{\iffalse}\fi}

\newcommand\v@gobblelast@readobr
{%
  \toks0{\gobblelast@vobr}%
  \@gobblelast@iter
}

\newcommand\vgobblelast@readcbr
{%
  \afterassignment\v@gobblelast@readcbr\let\next=%
}%

\newcommand\gobblelast@vcbr{\iffalse{\fi}}

\newcommand\v@gobblelast@readcbr
{%
  \toks0{\gobblelast@vcbr}%
  \@gobblelast@iter
}

\newtoks\gobblelast@temptoks


\newcommand\voodoogobblelast@finish
{%
  \ifnum\@tempcnta>\z@
    \global\advance\@tempcnta\m@ne
   \else
    \expandafter\ifx\expandafter\gobblelast@vobr\the\toks\z@
      \edef\tmp{\noexpand\gobblelast@vobr\expandafter\unexpanded\expandafter{\expandafter\unexpanded\expandafter{\the\gobblelast@temptoks}}\the\gobblelast@toks}
      \global\gobblelast@toks=\expandafter{\tmp}%
      \global\gobblelast@temptoks{}
     \else
      \expandafter\ifx\expandafter\gobblelast@vcbr\the\toks\z@
        \edef\tmp{\noexpand\gobblelast@vcbr\expandafter\unexpanded\expandafter{\expandafter\unexpanded\expandafter{\the\gobblelast@temptoks}}\the\gobblelast@toks}
        \global\gobblelast@toks=\expandafter{\tmp}%
        \global\gobblelast@temptoks{}
       \else
        \global\gobblelast@temptoks=\expandafter{\the\toks\expandafter\z@\the\gobblelast@temptoks}%
      \fi
    \fi
  \fi
  \endgroup
}

第一个版本\gobblelast，将逐个删除标记，但括号必须特殊处理。我还没有找到一种方法将它们放入标记列表中，以致于它变得不平衡，因此它们被转换为 catcode 12。

测试

\newtoks\mytoks

\mytoks={a\foo#{this and that} \hhh  \fi \egroup  \bgroup}

\gobblelast{\mytoks}{10}

\showthe\mytoks

会给

> a\foo ##{this and .
l.274     \showthe\mytoks

但这些牙套已经不再是真正的牙套了。

如果你愿意接受以下递归定义弱令牌：

A弱令牌是

1. 任何单个令牌不是括号或
2. 任何列表弱令牌括在括号内。

然后\weakgobblelast会保留括号，但括号内的所有内容将被视为一令牌。因此，

\newtoks\mytoks

\mytoks={a\foo#{this and that} \hhh  \fi \egroup  \bgroup}

\weakgobblelast{\mytoks}{6}

\showthe\mytoks

会给

> a\foo ##.
l.274     \showthe\mytoks

如果你真的想用“真正的”不平衡括号组来施展巫术，那么它\voodoogobblelast就是为你准备的。它将创建一个“安全包”的令牌，全面扩张后重新创建截断的令牌列表包括可能不平衡的牙套...

小心轻放！ ;-）

下列

\newtoks\mytoks

\mytoks={a\foo#{this and that} \hhh  \fi \egroup  \bgroup}

\voodoogobblelast{\mytoks}{10}

\showthe\mytoks

\mytoks{\newcommand\foo{\newcommand\bar[1]{\baz\space\quux}}}

\voodoogobblelast{\mytoks}{4}

\showthe\mytoks

\expandafter\edef\expandafter\next\expandafter{\the\mytoks}}}

\show\next

\next

\show\foo

会给

> \unexpanded {a\foo ##}\gobblelast@vobr \unexpanded {this and }.
l.274     \showthe\mytoks

> \unexpanded {\newcommand \foo }\gobblelast@vobr \unexpanded {\newcommand \bar [1]}\gobblelast@vobr \unexpanded {\baz }.
l.280     \showthe\mytoks

> \next=macro:
->\newcommand \foo {\newcommand \bar [1]{\baz }}.
l.284     \show\next

> \foo=\long macro:
->\newcommand \bar [1]{\baz }.
l.288     \show\foo

我隐约感觉到我犯下了一些致命的错误，但无论如何......

Answer 4

我希望保留空格和括号，但我发现 Stephan Lehmke 的解决方案太长了。

\documentclass{article}
\makeatletter
\newtoks\giventoks
\newtoks\@temptokenb
\def\removelasttoken{%
  \let\origbgroup\bgroup
  \let\bgroup\@undefined
  \@temptokena{}\@temptokenb{}%
  \expandafter\removelasttoken@a\the\giventoks\removelasttoken
  \giventoks=\@temptokenb
  \let\bgroup\origbgroup
}
\def\removelasttoken@a{\futurelet\next\removelasttoken@b}
\def\removelasttoken@b{%
  \ifx\next\@sptoken
    \expandafter\removelasttoken@c
  \else
    \expandafter\removelasttoken@e
  \fi
}
\@namedef{removelasttoken@c} {\futurelet\next\removelasttoken@d}
\def\removelasttoken@d{%
  \ifx\next\removelasttoken
    \@temptokenb\expandafter{\the\expandafter\@temptokenb\the\@temptokena}%
    \expandafter\removelasttoken@e
  \else
    \@temptokenb\expandafter
      {\the\expandafter\@temptokenb\the\expandafter\@temptokena\space}%
    \@temptokena{}%
    \expandafter\removelasttoken@a
  \fi
}
\def\removelasttoken@e#1{%
  \ifx\removelasttoken#1\else
    \@temptokenb\expandafter{\the\expandafter\@temptokenb\the\@temptokena}%
    \@temptokena{#1}%
    \edef\reserved@a{%
      \@temptokena{\ifx\next\origbgroup
        {\the\@temptokena}\else\the\@temptokena\fi}%
    }%
    \reserved@a
    \expandafter\removelasttoken@a
  \fi
}
\makeatother
% Example:
\begin{document}
\def\showout{\typeout{\the\giventoks}}
\giventoks{a\foo{#} {{this and that}} \hhh \fi \ifx \egroup  \bgroup \undefined}
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\removelasttoken\showout
\end{document}

该示例产生

a\foo {##} {{this and that}} \hhh \fi \ifx \egroup \bgroup 
a\foo {##} {{this and that}} \hhh \fi \ifx \egroup 
a\foo {##} {{this and that}} \hhh \fi \ifx 
a\foo {##} {{this and that}} \hhh \fi 
a\foo {##} {{this and that}} \hhh 
a\foo {##} {{this and that}}<space>
a\foo {##} {{this and that}}
a\foo {##}<space>
a\foo {##}
a\foo 

使用 eTeX，我们还可以做到

% \gobblelasttoken<cmd>
\def\gobblelasttoken#1{%
  \let\origbgroup\bgroup
  \let\bgroup\@undefined
  \@temptokena{}\@temptokenb{}%
  \expandafter\removelasttoken@a#1\gobblelasttoken
  \let\bgroup\origbgroup
  \edef#1{\the\@temptokenb}%
}

\removelasttoken@a其余宏保持不变。

例子：

\begin{document}
\def\showout{\typeout{\unexpanded\expandafter{\giventoks}}}
\edef\giventoks{\unexpanded{a\foo{#} {{this and that}} \hhh \fi \ifx
  \egroup\bgroup \undefined}}
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\gobblelasttoken\giventoks\showout
\end{document}