Skip to main content
 首页 » 编程设计

python之AWS lambda : Using executable with python

2025年02月15日10exmyth

你好this tutorial几乎准确地描述了我需要做什么并很好地解释了它,但它对我不起作用。 我搜索了类似的案例,甚至发现其他人也遇到了麻烦,但他们的解决方案对我没有帮助。

我稍微更改了文章中的代码,以便从同一个静态对象中提取元数据,这样我就可以轻松地在 lambda 中进行测试,而不必每次都将新文件上传到 s3 来触发该功能。 我还添加了一些输出,表明我应用了其他人的解决方案,例如添加“shell=True”并确保该文件确实是可执行的。

import logging 
import subprocess 
import boto3 
import botocore.session 
 
SIGNED_URL_EXPIRATION = 300     # The number of seconds that the Signed URL is valid 
DYNAMODB_TABLE_NAME = "metadata_test_db" 
DYNAMO = boto3.resource("dynamodb") 
TABLE = DYNAMO.Table(DYNAMODB_TABLE_NAME) 
logger = logging.getLogger('boto3') 
logger.setLevel(logging.INFO) 
 
 
def lambda_handler(event, context): 
    """ 
    :param event: 
    :param context: 
    """ 
 
    # Loop through records provided by S3 Event trigger 
    for s3_record in event['Records']: 
        logger.info("Working on new s3_record...") 
        # Extract the Key and Bucket names for the asset uploaded to S3 
        key = s3_record['s3']['object']['key'] 
        bucket = s3_record['s3']['bucket']['name'] 
        logger.info("Bucket: {} \t Key: {}".format(bucket, key)) 
        # Generate a signed URL for the uploaded asset 
        signed_url = get_signed_url(SIGNED_URL_EXPIRATION, bucket, key) 
        logger.info("Signed URL: {}".format(signed_url)) 
 
        # Launch MediaInfo 
        # Pass the signed URL of the uploaded asset to MediaInfo as an input 
        # MediaInfo will extract the technical metadata from the asset 
        # The extracted metadata will be outputted in XML format and 
        # stored in the variable xml_output 
        out2 = subprocess.check_output(["ls", "-l", "mediainfo"]) 
        logger.info(out2) 
        xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file"], shell=True) 
        logger.info("Output: {}".format(xml_output)) 
        #save_record(key, xml_output) 
 
def save_record(key, xml_output): 
    """ 
    Save record to DynamoDB 
 
    :param key:         S3 Key Name 
    :param xml_output:  Technical Metadata in XML Format 
    :return: 
    """ 
    logger.info("Saving record to DynamoDB...") 
    TABLE.put_item( 
       Item={ 
            'sryKey': key, 
            'technicalMetadata': xml_output 
        } 
    ) 
    logger.info("Saved record to DynamoDB") 
 
 
def get_signed_url(expires_in, bucket, obj): 
    """ 
    Generate a signed URL 
    :param expires_in:  URL Expiration time in seconds 
    :param bucket: 
    :param obj:         S3 Key name 
    :return:            Signed URL 
    """ 
    s3_cli = boto3.client("s3") 
    presigned_url = s3_cli.generate_presigned_url('get_object', Params={'Bucket': bucket, 'Key': obj}, 
                                                  ExpiresIn=expires_in) 
    return presigned_url 

来自 lambda 的 cloudwatch 错误日志返回以下内容

13:27:13 
START RequestId: cad3123b-f514-11e6-b8b1-45fa69956450 Version: $LATEST 
 
13:27:13 
[INFO]  2017-02-17T13:27:13.562Z    cad3123b-f514-11e6-b8b1-45fa69956450    Working on new s3_record... 
 
13:27:13 
[INFO]  2017-02-17T13:27:13.562Z    cad3123b-f514-11e6-b8b1-45fa69956450    Bucket: sourcebucket Key: HappyFace.jpg 
 
13:27:13 
[INFO]  2017-02-17T13:27:13.665Z    cad3123b-f514-11e6-b8b1-45fa69956450    Signed URL: https://s3.us-east-2.amazonaws.com/sourcebucket/HappyFace.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=300&X-Amz-Date=20170217T132713Z&X-Amz-SignedHeaders=host&X-Amz-Security-Token=FQoDYXdzEEMaDARR11pSPLcxi%2BkSZyL2AU2NjzOk37%2F2Ruwc33ZY5uN%2Ffg9O1c1awRcJf0qej4b29woEf%2BDhHsehkTr4WaKq19MxLL%2BdqmQBXArWXCYaGIcxdy 
 
13:27:13 
[INFO]  2017-02-17T13:27:13.683Z    cad3123b-f514-11e6-b8b1-45fa69956450    -rwxrwxrwx 1 slicer 497 9269401 Feb 17 09:51 mediainfo 
 
13:27:13 
Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255: CalledProcessError Traceback (most recent call last): File "/var/task/lambda_function.py", line 38, in lambda_handler xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file 
Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255: CalledProcessError 
Traceback (most recent call last): 
File "/var/task/lambda_function.py", line 38, in lambda_handler 
xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file"], shell=True) 
File "/usr/lib64/python2.7/subprocess.py", line 574, in check_output 
raise CalledProcessError(retcode, cmd, output=output) 
CalledProcessError: Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255 
 
 
13:27:13 
END RequestId: cad3123b-f514-11e6-b8b1-45fa69956450 
 
13:27:13 
REPORT RequestId: cad3123b-f514-11e6-b8b1-45fa69956450  Duration: 127.38 ms Billed Duration: 200 ms Memory Size: 512 MB Max Memory Used: 32 MB 

非常感谢任何提示。

请您参考如下方法:

所以对我来说,解决方案是添加 stderr=subprocess.STDOUT,就像建议的一样 here在对我的问题的评论中,还设置了 shell=False。 它不适用于 shell=True。 该提示来自 a comment至 this answer一个类似的问题,它“神奇地”与这个设置一起工作