S3 Uploader presigned url timestamp issue

[BrandonIngalls]

S3 Uploader presigned url timestamp issue

[ColdHeat]

Hi there, I believe you ran into an issue but I don't think your proposed solution is the right solution.

68da009

truncated_timestamp is from time.time() controlled by the system and returns a linux timestamp which isn't affected by timezones.

Are you sure that your system time is set correctly?

[BrandonIngalls]

The issue is not the use of time.time, that will return the epoch time which like you said is not effected by timezones.

The issue is in the use of datetime.fromtimestamp

fromtimestamp(...) method of builtins.type instance
    timestamp[, tz] -> tz's local time from POSIX timestamp.

fromtimestamp returns a local datetime object.

Where utcfromtimestamp returns a UTC datetime object.

utcfromtimestamp(...) method of builtins.type instance
    Construct a naive UTC datetime from a POSIX timestamp.

The difference in what these two datetime objects are is causing the boto project to generate urls with their dates set to the wrong UTC time.

#! /usr/bin/env python3

import time
import boto3
from botocore.client import Config
from freezegun import freeze_time
from datetime import datetime


SIGV4_TIMESTAMP = "%Y%m%dT%H%M%SZ"


def main() -> None:
    s3 = boto3.client(
        "s3",
        config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
        aws_access_key_id="AAAAAAAAAAAA",
        aws_secret_access_key="BBBBBBBBBBBBBBBBBBB",
        endpoint_url="https://localhost",
        region_name="us-east-1",
    )

    timestamp = int(time.time())

    broken_local_time = datetime.fromtimestamp(timestamp)
    correct_utc_time = datetime.utcfromtimestamp(timestamp)

    print(f" broken time: {broken_local_time.strftime(SIGV4_TIMESTAMP)}")
    print(f"correct time: {correct_utc_time.strftime(SIGV4_TIMESTAMP)}")

    with freeze_time(broken_local_time):
        url = s3.generate_presigned_url(
            "get_object",
            Params={
                "Bucket": "bucket-name",
                "Key": "/path/to/file.txt",
                "ResponseContentDisposition": "attachment; filename={}".format(
                    "file.txt"
                ),
                "ResponseCacheControl": "max-age=3600",
            },
            ExpiresIn=3600,
        )

    print(f"{url=}")


if __name__ == "__main__":
    main()

[~]$ date; python main.py
Wed Nov 29 07:42:32 CST 2023
 broken time: 20231129T074232Z
correct time: 20231129T134232Z
url='https://localhost/bucket-name//path/to/file.txt?response-content-disposition=attachment%3B%20filename%3Dfile.txt&response-cache-control=max-age%3D3600&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AAAAAAAAAAAA%2F20231129%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231129T074232Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=94f1c638bf0c805db82c66cb6f17cdba84bea722ad5b0ccd87b8465dd0ce7242'

Note how my local time is in US/Central, Wed Nov 29 07:42:32 CST 2023

and that CTFd generated a presigned url with X-Amz-Date=20231129T074232Z (2023 11 29 07:42:32Z)

To the S3 service that Date param MUST be presented as a UTC value, and the correct UTC time when I ran this script was (2023 11 29 13:42:32Z)

So when I attempt to navigate to that URL I get an access denied because the url CTFd generated for me had already expired many hours before.

07:42:32Z (incorrectly generated because of fromtimestamp)
13:42:32Z (what should have been generated if utcfromtimestamp had been used)

[ColdHeat]

Thanks for the POC. I think this is probably an issue with freezegun or something which is apparently no longer being mainatined. spulec/freezegun#511