算法包

Question 1

我假设你正在使用这个algorithmicx包，尽管我没有看到它在任何地方被导入。你可以看看文档。您不应该手动缩进或加粗。例如，您的为了循环将使用以下方式标记：

\For{step {$h=1, 2, \ldots, H$}}\Comment{(policy improvement)} \label{line:pis-start} \STATE Update policy $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot) \cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$ \label{line:policy-improvement} \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$. \STATE Observe reward $r_h(s_h^k,a^k_h)$, get next state $s^k_{t+1}$. \label{line:pis-end} \EndFor

\Comment另请注意右对齐注释的使用。

~~这应该可以解决您的缩进问题，也可能解决换行问题。我将算法标记的重写留给您，我们可以看看还需要做什么。~~

在下面的评论中，您表示您实际使用的algorthimic语法和文档略有不同这里。你仍然不应该手动缩进或加粗。以下是为了从上面循环：

\FOR[policy improvement]{step {$h=1, 2, \ldots, H$}} \label{line:pis-start}
  \STATE Update policy $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot) \cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$ \label{line:policy-improvement}
  \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \STATE Observe reward $r_h(s_h^k,a^k_h)$, get next state $s^k_{t+1}$.  \label{line:pis-end}
\ENDFOR

algorithmic所有命令均使用大写字母（也有一个\COMMENT命令，但它不适用于循环/流控制结构，因此对于这些结构，您需要在语句中使用可选参数，如上所示）。

目前还不清楚您希望通过\vspace算法中的命令实现什么目标。

Answer

我假设你正在使用这个algorithmicx包，尽管我没有看到它在任何地方被导入。你可以看看文档。您不应该手动缩进或加粗。例如，您的为了循环将使用以下方式标记：

\For{step {$h=1, 2, \ldots, H$}}\Comment{(policy improvement)} \label{line:pis-start} \STATE Update policy $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot) \cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$ \label{line:policy-improvement} \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$. \STATE Observe reward $r_h(s_h^k,a^k_h)$, get next state $s^k_{t+1}$. \label{line:pis-end} \EndFor

\Comment另请注意右对齐注释的使用。

~~这应该可以解决您的缩进问题，也可能解决换行问题。我将算法标记的重写留给您，我们可以看看还需要做什么。~~

在下面的评论中，您表示您实际使用的algorthimic语法和文档略有不同这里。你仍然不应该手动缩进或加粗。以下是为了从上面循环：

\FOR[policy improvement]{step {$h=1, 2, \ldots, H$}} \label{line:pis-start}
  \STATE Update policy $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot) \cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$ \label{line:policy-improvement}
  \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \STATE Observe reward $r_h(s_h^k,a^k_h)$, get next state $s^k_{t+1}$.  \label{line:pis-end}
\ENDFOR

algorithmic所有命令均使用大写字母（也有一个\COMMENT命令，但它不适用于循环/流控制结构，因此对于这些结构，您需要在语句中使用可选参数，如上所示）。

目前还不清楚您希望通过\vspace算法中的命令实现什么目标。

Question 2

要点是：您必须考虑如何构造行，以便换行符出现在合理的位置。这不是可以自动发生的。作为起点，您会在下面找到您的算法，一次使用包排版，algorithmic一次使用algorithmicx包（由隐式加载algpseudocode）。后者比前者更灵活一些。

算法包

\documentclass[twoside]{article}
\usepackage{aistats2021}
\usepackage{algorithm}
\usepackage{algorithmic}
\newcommand\cA{\mathcal A}
\begin{document}

\begin{algorithm}[t]
\caption{\label{Algorithm-TS} Algorithm }
\begin{algorithmic}[1]
\STATE Initialize $\{Q^0_h\}_{h=1}^H$ as zero functions\\
 and $\{\pi^0_h\}_{h=1}^H$ as uniform distributions on $\cA$. \label{line:winit}
\FOR{episode $k=1,2,\ldots, K$}
\STATE Receive the initial state $s_1^k$.
\FOR[policy improvement]{step {$h=1, 2, \ldots, H$}}\label{line:pis-start}
  \STATE $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot)\cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$\label{line:policy-improvement}\\
  \hfill\COMMENT{policy update}
  \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \STATE Observe reward $r_h(s_h^k,a^k_h)$,\label{line:pis-end}\\
   \hfill get next state $s^k_{t+1}$.
  \ENDFOR
\STATE Initialize $V^k_{H+1}$ as a zero function.\\
\FOR[policy evaluation]{step $h=H, H-1,\ldots, 1$} \label{line:pes-start}
\STATE $\Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot) \leftarrow  \phi_h^{k}(\cdot, \cdot)^\top(\hat{\theta}^k_h + \xi^{k,j}_h)$ for $j \in [M]$.\label{line:PVh_plus_1_tilde_perturbed}
\STATE $Q^k_h(\cdot,\cdot) \leftarrow \min \{\max_{j \in [M]} \Tilde{r}^{k,j}_h(\cdot,\cdot) + \mbox{}$\\
\hfill$\max_{j \in [M]} \Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot), H -h +\}^{+}$\label{line:min-max-for-Q}
\STATE $V^k_h(\cdot) \leftarrow \langle Q^k_h(\cdot,\cdot), \pi^k_h(\cdot\,|\,\cdot) \rangle_{\cA}$\label{line:pes-end}
  \ENDFOR
  \ENDFOR
\end{algorithmic}
\end{algorithm}

\end{document}

algorithmicx/algpseudode 包

\documentclass[twoside]{article}
\usepackage{aistats2021}
\usepackage{algorithm}
\usepackage{algpseudocode}
\newcommand\cA{\mathcal A}
\begin{document}

\begin{algorithm}[t]
\caption{\label{Algorithm-TS} Algorithm }
\begin{algorithmic}[1]
\renewcommand\algorithmicindent{1em}
\State Initialize $\{Q^0_h\}_{h=1}^H$ as zero functions
\Statex and $\{\pi^0_h\}_{h=1}^H$ as uniform distributions on $\cA$. \label{line:winit}
\For{episode $k=1,2,\ldots, K$}
\State Receive the initial state $s_1^k$.
\For{step {$h=1, 2, \ldots, H$}} \label{line:pis-start}
\Statex\Comment{policy improvement}
  \State $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot)\cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$\label{line:policy-improvement}
  \Statex\Comment policy update
  \State Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \State Observe reward $r_h(s_h^k,a^k_h)$,  \label{line:pis-end}
  \Statex \hfill get next state $s^k_{t+1}$.
  \EndFor
\State Initialize $V^k_{H+1}$ as a zero function.
\For{step $h=H, H-1,\ldots, 1$} \label{line:pes-start}
\Statex\Comment{policy evaluation}
\State $\Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot) \leftarrow  \phi_h^{k}(\cdot, \cdot)^\top(\hat{\theta}^k_h + \xi^{k,j}_h)$ for $j \in [M]$.\label{line:PVh_plus_1_tilde_perturbed}
\State $Q^k_h(\cdot,\cdot) \leftarrow \min \{\max_{j \in [M]} \Tilde{r}^{k,j}_h(\cdot,\cdot) + \mbox{}$
\Statex\hfill$\max_{j \in [M]} \Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot), H -h +\}^{+}$\label{line:min-max-for-Q}
\State $V^k_h(\cdot) \leftarrow \langle Q^k_h(\cdot,\cdot), \pi^k_h(\cdot\,|\,\cdot) \rangle_{\cA}$\label{line:pes-end}
  \EndFor
  \EndFor
\end{algorithmic}
\end{algorithm}

\end{document}

Answer

要点是：您必须考虑如何构造行，以便换行符出现在合理的位置。这不是可以自动发生的。作为起点，您会在下面找到您的算法，一次使用包排版，algorithmic一次使用algorithmicx包（由隐式加载algpseudocode）。后者比前者更灵活一些。

算法包

\documentclass[twoside]{article}
\usepackage{aistats2021}
\usepackage{algorithm}
\usepackage{algorithmic}
\newcommand\cA{\mathcal A}
\begin{document}

\begin{algorithm}[t]
\caption{\label{Algorithm-TS} Algorithm }
\begin{algorithmic}[1]
\STATE Initialize $\{Q^0_h\}_{h=1}^H$ as zero functions\\
 and $\{\pi^0_h\}_{h=1}^H$ as uniform distributions on $\cA$. \label{line:winit}
\FOR{episode $k=1,2,\ldots, K$}
\STATE Receive the initial state $s_1^k$.
\FOR[policy improvement]{step {$h=1, 2, \ldots, H$}}\label{line:pis-start}
  \STATE $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot)\cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$\label{line:policy-improvement}\\
  \hfill\COMMENT{policy update}
  \STATE Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \STATE Observe reward $r_h(s_h^k,a^k_h)$,\label{line:pis-end}\\
   \hfill get next state $s^k_{t+1}$.
  \ENDFOR
\STATE Initialize $V^k_{H+1}$ as a zero function.\\
\FOR[policy evaluation]{step $h=H, H-1,\ldots, 1$} \label{line:pes-start}
\STATE $\Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot) \leftarrow  \phi_h^{k}(\cdot, \cdot)^\top(\hat{\theta}^k_h + \xi^{k,j}_h)$ for $j \in [M]$.\label{line:PVh_plus_1_tilde_perturbed}
\STATE $Q^k_h(\cdot,\cdot) \leftarrow \min \{\max_{j \in [M]} \Tilde{r}^{k,j}_h(\cdot,\cdot) + \mbox{}$\\
\hfill$\max_{j \in [M]} \Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot), H -h +\}^{+}$\label{line:min-max-for-Q}
\STATE $V^k_h(\cdot) \leftarrow \langle Q^k_h(\cdot,\cdot), \pi^k_h(\cdot\,|\,\cdot) \rangle_{\cA}$\label{line:pes-end}
  \ENDFOR
  \ENDFOR
\end{algorithmic}
\end{algorithm}

\end{document}

algorithmicx/algpseudode 包

\documentclass[twoside]{article}
\usepackage{aistats2021}
\usepackage{algorithm}
\usepackage{algpseudocode}
\newcommand\cA{\mathcal A}
\begin{document}

\begin{algorithm}[t]
\caption{\label{Algorithm-TS} Algorithm }
\begin{algorithmic}[1]
\renewcommand\algorithmicindent{1em}
\State Initialize $\{Q^0_h\}_{h=1}^H$ as zero functions
\Statex and $\{\pi^0_h\}_{h=1}^H$ as uniform distributions on $\cA$. \label{line:winit}
\For{episode $k=1,2,\ldots, K$}
\State Receive the initial state $s_1^k$.
\For{step {$h=1, 2, \ldots, H$}} \label{line:pis-start}
\Statex\Comment{policy improvement}
  \State $\pi^k_h (\cdot \,|\, \cdot) \propto \pi^{k-1}_h(\cdot \,|\, \cdot)\cdot \exp\{\alpha Q^{k-1}_h(\cdot , \cdot)\}$\label{line:policy-improvement}
  \Statex\Comment policy update
  \State Take the action following $a^k_h\sim\pi^k_h(\cdot\,|\,s_h^k)$.
  \State Observe reward $r_h(s_h^k,a^k_h)$,  \label{line:pis-end}
  \Statex \hfill get next state $s^k_{t+1}$.
  \EndFor
\State Initialize $V^k_{H+1}$ as a zero function.
\For{step $h=H, H-1,\ldots, 1$} \label{line:pes-start}
\Statex\Comment{policy evaluation}
\State $\Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot) \leftarrow  \phi_h^{k}(\cdot, \cdot)^\top(\hat{\theta}^k_h + \xi^{k,j}_h)$ for $j \in [M]$.\label{line:PVh_plus_1_tilde_perturbed}
\State $Q^k_h(\cdot,\cdot) \leftarrow \min \{\max_{j \in [M]} \Tilde{r}^{k,j}_h(\cdot,\cdot) + \mbox{}$
\Statex\hfill$\max_{j \in [M]} \Tilde{P}_h\Tilde{V}^{k,j}_{h+1}(\cdot,\cdot), H -h +\}^{+}$\label{line:min-max-for-Q}
\State $V^k_h(\cdot) \leftarrow \langle Q^k_h(\cdot,\cdot), \pi^k_h(\cdot\,|\,\cdot) \rangle_{\cA}$\label{line:pes-end}
  \EndFor
  \EndFor
\end{algorithmic}
\end{algorithm}

\end{document}

算法包

答案1

答案2

算法包

algorithmicx/algpseudode 包

相关内容