☰

绘制 softmax 激活函数

tag-icon • tag-icon tikz-pgf pgfplots
绘制 softmax 激活函数

在下面的 MSE 中，我定义了一个函数logsumexp 来[declare function={logsumexp(\x)=\log(\sum{\exp^{\x_i}});}]帮助绘制 softmax 激活函数。

当我使用该函数添加情节时，\addplot[blue,smooth] {exp(x) /logsumexp(x))};一切都乱了。

MSE：（\addplot为softmax函数注释掉的行）

\documentclass[11pt]{article}

\usepackage{subfigure}
\usepackage{pgfplots}
\usepackage[top=3cm,left=3cm,right=3cm,bottom=3cm]{geometry}
% Scriptsize axis style.
\pgfplotsset{every axis/.append style={tick label style={/pgf/number format/fixed},font=\scriptsize,ylabel near ticks,xlabel near ticks,grid=major}}

\pgfplotsset{compat=1.16}

\begin{document}
\begin{figure}[t!]
    \centering
    \subfigure[sigmoid activation function.]{
            \begin{tikzpicture}[declare function={sigma(\x)=1/(1+exp(-\x));}]
            \begin{axis}[width=5.5cm,height=6cm,ylabel=$\sigma(z)$,xlabel=$z$,ymin=0,ymax=1.25,xmin=-5,xmax=5]
                \addplot[blue,smooth] {1/(1+exp(-x))};
            \end{axis}
        \end{tikzpicture}
    }
    \subfigure[Softmax activation function. ]{
        \begin{tikzpicture}[declare function={logsumexp(\x)=\log(\sum{\exp^{\x_i}});}]
            \begin{axis}[width=5.5cm,height=6cm,ylabel=$ \sigma(z)_j$,xlabel=$z$,ymin=-1.25,ymax=1.25,xmin=-5,xmax=5]
               %\addplot[blue,smooth] {exp(x) /logsumexp(x))};

            \end{axis}
        \end{tikzpicture}
    }
        \caption[Activation functions.]{Sigmoid and Softmax activation functions}
        \label{fig:sigmoid-tanh}
\end{figure}
logsumexp function: $logsumexp(x)=\log(\sum{\exp^{x_i}})$
\end{document}

在此处输入图片描述

当 \addplot 取消注释时，一切都乱了。我错过了什么？

Check that your $'s match around math expressions. If they do, then you've probably used a symbol in normal text that needs to be in math mode. Symbols such as subscripts ( _ ), integrals ( \int ), Greek letters ( \alpha, \beta, \delta ), and modifiers (\vec{x}, \tilde{x} ) must be written in math mode. See the full list here.If you intended to use mathematics mode, then use $ … $ for 'inline math mode', $$ … $$ for 'display math mode' or alternatively \begin{math} … \end{math}.

编辑

给出一些具有 值 的例子x。

import numpy as np
x = [1.2, 2.5, 3.1, 4.4, 1.6, 2.4, 3.6]
np.exp(x) / np.sum(np.exp(x))
array([0.01933382, 0.07094152, 0.12926387, 0.47430749, 0.02884267,
       0.06419054, 0.21312009])

答案1

至于你的问题，正如评论中提到的，\exp是\log命令排版这些函数，您可能希望使用内置函数exp和ln。同样，\sum是一个排版求和符号的命令，但与前面的情况不同，没有内置函数。所以必须构建它。下面包含一个示例。（如果这不是您想要的精确函数，您需要提供一个精确的解释，以通用术语定义该函数，即使用数学语言而不是某些 Python 代码。）

\documentclass[11pt]{article}
\usepackage[top=3cm,left=3cm,right=3cm,bottom=3cm]{geometry}
\usepackage{amsmath}
\DeclareMathOperator{\logsumexp}{logsumexp}
\usepackage{subcaption}
\usepackage{pgfplots}
\pgfplotsset{compat=1.16}
\pgfplotsset{every axis/.append style={tick label style={/pgf/number format/fixed},font=\scriptsize,ylabel near ticks,xlabel near ticks,grid=major}}

\pgfmathdeclarefunction{sumexp}{3}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\pgfmathsetmacro{\myx}{#1}%
\pgfmathtruncatemacro{\myxmin}{#2}%
\pgfmathtruncatemacro{\myxmax}{#3}%
\pgfmathsetmacro{\mysum}{0}%
\pgfplotsforeachungrouped\XX in {\myxmin,...,\myxmax}%
{\pgfmathsetmacro{\mysum}{\mysum+exp(\XX)}}%
\pgfmathparse{\mysum+exp(#1)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%

\begin{document}
\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.45\textwidth}
            \begin{tikzpicture}[declare function={sigma(\x)=1/(1+exp(-\x));}]
            \begin{axis}[width=5.5cm,height=6cm,ylabel=$\sigma(z)$,xlabel=$z$,ymin=0,ymax=1.25,xmin=-5,xmax=5]
                \addplot[blue,smooth] {1/(1+exp(-x))};
            \end{axis}
        \end{tikzpicture}
     \caption{Sigmoid activation function.}
    \end{subfigure} 
    \begin{subfigure}[t]{0.45\textwidth}    
        \begin{tikzpicture}
            \begin{axis}[width=5.5cm,height=6cm,ylabel=$\sigma(z)_j$,
            xlabel=$z$,     xmin=-5,xmax=5]
               \addplot[blue,domain=-5:5,samples=51] 
               {exp(x)/sumexp(x,-4,0)};
            \end{axis}
        \end{tikzpicture}
      \caption{Softmax activation function.}
     \end{subfigure}  
   \caption[Activation functions.]{Sigmoid and Softmax activation functions}
   \label{fig:sigmoid-tanh}
\end{figure}
\end{document}

在此处输入图片描述

答案2

%for Softmax function, place this command before \begin{document}.
\pgfmathdeclarefunction{sumexp}{3}{%
  \begingroup%
  \pgfkeys{/pgf/fpu}% "/pgf/fpu/output format=fixed" removed
  \pgfmathsetmacro{\myx}{#1}%
  \pgfmathtruncatemacro{\myxmin}{#2}%
  \pgfmathtruncatemacro{\myxmax}{#3}%
  \pgfmathsetmacro{\mysum}{0}%
  \pgfplotsforeachungrouped\XX in {\myxmin,...,\myxmax}%
    {\pgfmathsetmacro{\mysum}{\mysum+exp(\XX)}}%
  \pgfmathparse{\mysum+exp(#1)}%
  \pgfmathfloattofixed\pgfmathresult%  added
  \pgfmathsmuggle\pgfmathresult\endgroup%
}%

定义上述 sumexp 后，您可以使用 sumexp 命令绘制 Softmax 激活函数。

\begin{figure}
    \centering      
      \begin{tikzpicture}[declare function= 
       {Softmax(\x)=exp(x)/sumexp(x,-5,0);}]
      \begin{axis}%
        [
        xlabel = $x$,
        ylabel = $y$,
        %grid=major,     
        xmin=-5,
        xmax=5,
        axis x line=middle,
        ymax=1, samples=100,
        axis y line=middle,
        domain=-5:5}     
        ]
        \addplot[blue]  {Softmax(x)};
      \end{axis}
    \end{tikzpicture}        
    \caption{Softmax Function}
    \label{fig:softmax_function}
\end{figure}

您将获得以下输出：

Softmax 函数

我知道图像可能不是完全适合，但网格、颜色等可以由您选择。希望这会有所帮助。

相关内容