☰

通过 cron tab 运行脚本时出现问题

tag-icon • tag-icon command-line scripts python cron
通过 cron tab 运行脚本时出现问题

我是 Ubuntu 新手，第一次编写 crontab，或者使用 Nano 编辑器。基本上我可以通过 crontab 来触发一个简单的 python 脚本，但是不是需要安排的实际 Python 脚本。以下是我所做的。

我可以通过 crontab 运行这个简单的 python 脚本（称为 testChronScript.py）：

#!/usr/bin/python
# -*- coding: utf-8 -*-

print "TESTING"
file_name = "test_outputfile.txt"
output = open(file_name, "w")
output.write("TEST")
print "DONE"

我设置 cron 选项卡的方式以及生成的输出如下所示：

justin@JBot:~/PycharmProjects/Quant_Local$ sudo nano runScripts.cron 
justin@JBot:~/PycharmProjects/Quant_Local$ crontab runScripts.cron
justin@JBot:~/PycharmProjects/Quant_Local$ crontab -l
41 14 * * * python testChronScript.py  > /dev/pts/19 
justin@JBot:~/PycharmProjects/Quant_Local$ TESTING
DONE

这个python脚本将会按照指定的时间在下午2:41运行，并在目录中创建文件。

现在实际脚本我正在尝试通过 crontab 设置从 wikipedia 抓取数据并将其保存到数据库。代码如下：

#!/usr/bin/python
# -*- coding: utf-8 -*-

# Author: Justin Dano 8/6/2016
# This script was inspired by Michael Halls-Moore articles on Quantstart.com

import datetime
import lxml.html
from urllib2 import urlopen
from math import ceil
from SharedFunctionsLib import *


def scrape_sp500_symbols():
    """
    Scrape S&P500 symbols from Wikipedia page
    :return: List of current SP500 symbols
    """
    timestamp = datetime.datetime.utcnow()

    # Use libxml to scrape S&P500 ticker symbols
    page = urlopen('http://en.wikipedia.org/wiki/List_of_S%26P_500_companies')
    page = lxml.html.parse(page)
    symbols_list = page.xpath('//table[1]/tr')[1:]

    # Obtain the ticker symbol, name, and sector information 
    # for each row in the S&P500 constituent table
    ticker_symbols = []
    for symbol in symbols_list:
        tds = symbol.getchildren()
        sd = {'ticker': tds[0].getchildren()[0].text,
                'name': tds[1].getchildren()[0].text,
                'sector': tds[3].text}

        # Map ticker information to the columns of our database table
        # The first value (2) represents the status id, which is set to live by default
        ticker_symbols.append((2, sd['ticker'], sd['name'],
            sd['sector'], timestamp, timestamp))

    return ticker_symbols


def filter_symbols(symbols):
    """
    If we are updating our symbols table, we do not want to
    add duplicate symbols, so here we filter out companies
    that already exist in the database.
    :param symbols: The list of symbols scraped from Wikipedia
    :return: List of symbols not yet represented in database
    """
    new_symbols = [] 
    unique_symbols = set() 

    # Attempt to get any existing ticker data
    data = retrieve_db_tickers(con)

    # Collect a set of existing tickers
    for symbol in data:
        unique_symbols.add(symbol[1])

    # Now add any additional symbols not yet included in the
    # database from the SP500 wiki page
    for s in symbols:
        if s[0] not in unique_symbols:
            print(str(s[2]) + " will be added to the database!")
            new_symbols.append(s)

    return new_symbols


def insert_sp500_symbols(symbols):
    """
    Insert any new S&P500 symbols (that do not already belong)
    into the MySQL database.
    :param symbols: List of tuples where each tuple is data for a specific company
    """
    # Create the insert query
    column_fields = "status_id, ticker, name, sector, created_date, last_updated_date"
    insert_fields = ("%s, " * 6)[:-2]
    query_string = "INSERT INTO symbol (%s) VALUES (%s)" % (column_fields, insert_fields)

    # Insert symbol data into the database for every symbol
    with con: 
        cur = con.cursor()
        # This line avoids the MySQL MAX_PACKET_SIZE
        # It chunks the inserts into sets of 100 at a time
        for i in range(0, int(ceil(len(symbols) / 100.0))):
            cur.executemany(query_string, symbols[i*100:(i+1)*100-1])


if __name__ == "__main__":
    con = get_db_connection()

    # 1.Scrape ticker data for the current companies existing in the S&P500 index from Wikipedia
    symbols = scrape_sp500_symbols()

    # 2.Filter out pre-existing data that may already belong in our database
    filtered_symbols = filter_symbols(symbols)

    # 3.Insert company ticker data into our MySQL database
    insert_sp500_symbols(filtered_symbols)

现在输出如下所示：

justin@JBot:~/PycharmProjects/Quant_Local$ sudo nano runScripts.cron
justin@JBot:~/PycharmProjects/Quant_Local$ crontab runScripts.cron
justin@JBot:~/PycharmProjects/Quant_Local$ crontab -l
51 14 * * * python obtainSymbols.py  > /dev/pts/19 
justin@JBot:~/PycharmProjects/Quant_Local$

基本上，您应该看到一些额外的输出（以及保存到数据库的新条目。）该脚本似乎尚未执行我不知道为什么！

这是证明目录中文件存在的目录，以及它们相应的权限。

drwxrwxr-x 3 justin justin  4096 Sep 27 14:50 .
drwxrwxr-x 4 justin justin  4096 Sep 24 14:54 ..
-rw-rw-r-- 1 justin justin  3504 Sep 25 15:02 obtainSymbols.py
-rw-r--r-- 1 root   root      52 Sep 27 14:50 runScripts.cron
-rw-rw-r-- 1 justin justin  5009 Sep 27 12:17 SharedFunctionsLib.py
-rw-rw-r-- 1 justin justin   174 Sep 25 16:56 testChronScript.py

关于为什么我的 crontab 不会触发我的 possessSymbols.py 脚本，您有什么建议吗？或者，关于如何安排此 python 脚本运行（基本上每天运行）的任何其他建议都会很有帮助！

谢谢你的时间。

答案1

输出重定向到文本文件是不正确的。cron使用 运行脚本sh，因此您的cron行应如下所示：

41 14 * * * python testChronScript.py >> /dev/pts/19 2>&1

运行脚本时，cron命令行上不会有任何输出。相反，请使用以下命令检查输出文件的内容：

cat /dev/pts/19

此外，以 root 身份创建该文件没有任何意义runScripts.cron。

相关内容