有没有办法制作下面的图表
通过修改下面的代码（tikzpicture）？
\documentclass{standalone}
\usepackage{tikz}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{document}
\def\layersep{3cm}
\tikzstyle{block} = [rectangle, draw,
text width=8em, text centered, rounded corners, minimum height=4em]
\tikzstyle{line} = [draw, -latex]
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[ node distance = 7em, auto, thick]
\node[block](Agent) {Agent};
\node [block, below of=Agent] (Environment) {Environment};
%\node [block, right of=PEnvironment] (MEnvironment) {M Env};
\path [line] (Agent.0) --++ (7em,0em) |- node [near start]{$a_t$} (Environment.-10);
\path [line] (Environment.200) --++ (-7em,0em) |- node [near start]
{ \rotatebox{90}{$s_t \rightarrow s_{t+1}$}} (Agent.165);
\path [line] (Environment.180) --++ (-5.25em,0em) |- node [near start, right]
{\rotatebox{90}{$r_t \rightarrow r_{t+1}$}} (Agent.185);
\end{tikzpicture}
\begin{tikzpicture}[
shorten >=1pt,->,
draw=black!50,
node distance=\layersep,
every pin edge/.style={<-,shorten <=1pt},
neuron/.style={circle, fill=gray!25, minimum size=17pt, inner sep=0pt},
input neuron/.style={neuron, fill=gray!10},
hidden neuron/.style={neuron, fill=gray!20},
output neuron/.style={neuron, fill=gray!30},
annot/.style={text width=4em, text centered}
]
% Draw the input layer nodes
\foreach \name / \y in {1,...,3}
% This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
\node[input neuron] (I-\name) at (0,-\y) {$x_\y$};
% Draw the hidden layer nodes
\foreach \name / \y in {1,...,3}
\path[yshift=0cm]
node[hidden neuron] (H-\name) at (\layersep,-\y cm) {$f(z_\y)$};
% Draw the output layer node
%\node[output neuron,pin={[pin edge={->}]right:$\hat{y}$}, right of=H-3] (O) {};
\node[output neuron,pin={[pin edge={->}]right:}, right of=H-2] (O) {$h_{\hat{w},\hat{b}}(x)$};
% Connect every node in the input layer with every node in the
% hidden layer.
\foreach \source in {1,...,3}
\foreach \dest in {1,...,3}
\path (I-\source) edge (H-\dest);
% Connect every node in the hidden layer with the output layer
\foreach \source in {1,...,3}
\path (H-\source) edge (O);
% Annotate the layers
\node[annot,above of=H-1, node distance=1.5cm] (hl) {Hidden Layer ($l=2$)};
\node[annot,left of=hl] {Input Layer ($l=1$)};
\node[annot,right of=hl] {Output Layer ($l=3$)};
\end{tikzpicture}
\end{document}
一旦执行上述代码，两个网络就会并排出现。如果我能将右侧插图中描绘的网络放在左侧网络内（具体来说是标有 Agent 的框内），那就太好了。
答案1
由于某种原因，我的硬盘上有一段非常相似的代码，我稍加修改后发布到这里，以便提供一些答案。即使它不能完全解决问题，至少也可以作为一个开始。
\documentclass[tikz,border=3mm]{standalone}
\usetikzlibrary{positioning,fit}
\begin{document}
\tikzset{%
neuron missing/.style={
draw=none,
scale=2,
text height=0.333cm,
execute at begin node=\color{black}$\vdots$
},
}
% The command \DrawNeuronalNetwork has a list as argument, each entry is a
% layer. each entry has the form
% Layer name/number of nodes/color/missing node/label/symbolic number
% where
% * layer name is, well, the name of the layer
% * number of nodes is the number of neurons in that layer (including the missing neuron)
% * color is the color of the layer
% * missing node denotes the index of the missing neuron
% * label denotes the label of the layer
% * symbolic number denotes the symbol that indicates how many neurons there are
% * node content
\newcommand{\DrawNeuronalNetwork}[2][]{
\xdef\Xmax{0}
\foreach \Layer/\X/\Col/\Miss/\Lab/\Count/\Content [count=\Y] in {#2}
{\pgfmathsetmacro{\Xmax}{max(\X,\Xmax)}
\xdef\Xmax{\Xmax}
\xdef\Ymax{\Y}
}
\foreach \Layer/\X/\Col/\Miss/\Lab/\Count/\Content [count=\Y] in {#2}
{\node[anchor=south] at ({2*\Y},{\Xmax/2+0.1}) {\Layer};
\foreach \m in {1,...,\X}
{
\ifnum\m=\Miss
\node [neuron missing] (neuron-\Y-\m) at ({2*\Y},{\X/2-\m}) {};
\else
\node [circle,fill=\Col!50,minimum size=1cm] (neuron-\Y-\m) at
({2*\Y},{\X/2-\m}) {\Content};
\ifnum\Y=1
\else
\pgfmathtruncatemacro{\LastY}{\Y-1}
\foreach \Z in {1,...,\LastX}
{
\ifnum\Z=\LastMiss
\else
\draw[->] (neuron-\LastY-\Z) -- (neuron-\Y-\m);
\fi
}
\fi
\fi
\ifnum\Y=1
\ifnum\m=\X
\draw [overlay] (neuron-\Y-\m) -- (state);
\else
\ifnum\m=\Miss
\else
\draw [overlay] (neuron-\Y-\m) -- (state);
\fi
\fi
\else
\fi
}
\xdef\LastMiss{\Miss}
\xdef\LastX{\X}
}
}
\begin{tikzpicture}[x=1.5cm, y=1.5cm,
>=stealth,font=\sffamily,nodes={align=center}]
\begin{scope}[local bounding box=T]
\path node[draw,minimum width=6em,minimum height=4em] (state) {State};
\begin{scope}[local bounding box=NN]
\DrawNeuronalNetwork{Input Layer/5/green/4///,
Hidden Layer/5/blue/4//11/,
Output Layer/4/red/3//11/}
\end{scope}
\path (NN.south) node[below]{Estimated parameter\\ $\theta$};
\path(NN.east) -- node[above]{Policy\\ $\Pi(\theta,a)$}++ (4em,0);
\end{scope}
\node[fit=(T),label={[anchor=north west]north west:Agent},inner sep=1em,draw]
(TF){};
\node[below=3em of TF,draw,inner sep=1em] (Env) {Environment};
\draw[<-] (TF.200) -- ++ (-1em,0) |- (Env.160) node[pos=0.45,right]{$r_t$};
\draw[<-] (TF.180) -- ++ (-2em,0) |- (Env.180) node[pos=0.45,left]{$s_t$};
\draw[->] (NN.east) -- ++ (7em,0)node[right]{$a_t$} |- (Env);
\end{tikzpicture}
\end{document}
