python converting string in localtime to UTC epoch timestamp

I have strings in YMD hms format that had the timezone stripped. But I know they are in Eastern time with daylight savings time.

I am trying to convert them into epoch timestamps for UTC time.

I wrote the following function:

def ymdhms_timezone_dst_to_epoch(input_str,  tz="US/Eastern"):
    print(input_str)
    dt = datetime.datetime.fromtimestamp(time.mktime(time.strptime(input_str,'%Y-%m-%d %H:%M:%S')))
    local_dt = pytz.timezone(tz).localize(dt)
    print(local_dt.strftime('%Y-%m-%d %H:%M:%S %Z%z'))
    utc_dt = local_dt.astimezone(pytz.utc)
    print(utc_dt.strftime('%Y-%m-%d %H:%M:%S %Z%z'))    
    e = int(utc_dt.strftime("%s"))
    print(e)
    return e

Given string `2015-04-20 21:12:07` this prints:

    2015-04-20 21:12:07
    2015-04-20 21:12:07 EDT-0400 #<- so far so good?
    2015-04-21 01:12:07 UTC+0000 #<- so far so good?
    1429596727

which looks ok up to the epoch timestamp. But http://www.epochconverter.com/epoch/timezones.php?epoch=1429596727 says it should mao to
Greenwich Mean Time Apr 21 2015 06:12:07 UTC.

What is wrong?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I have strings in YMD hms format that had the timezone stripped. But I know they are in Eastern time with daylight savings time.

A portable way is to use pytz:

#!/usr/bin/env python
from datetime import datetime
import pytz # $ pip install pytz

naive_dt = datetime.strptime('2015-04-20 21:12:07', '%Y-%m-%d %H:%M:%S')
tz = pytz.timezone('US/Eastern')
eastern_dt = tz.normalize(tz.localize(naive_dt))
print(eastern_dt)
# -> 2015-04-20 21:12:07-04:00

I am trying to convert them into epoch timestamps for UTC time.

timestamp = (eastern_dt - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()
# -> 1429578727.0

See Converting datetime.date to UTC timestamp in Python.

There are multiple issues in your code:

• time.mktime() may return a wrong result for ambiguous input time (50% chance) e.g., during “fall back” DST transition in the Fall
• time.mktime() and datetime.fromtimestamp() may fail for past/future dates if they have no access to a historical timezone database on a system (notably, Windows)
• localize(dt) may return a wrong result for ambiguous or non-existent time i.e., during DST transitions. If you know that the time corresponds to the summer time then use is_dst=True. tz.normalize() is necessary here, to adjust possible non-existing times in the input
• utc_dt.strftime("%s") is not portable and it does not respect tzinfo object. It interprets input as a local time i.e., it returns a wrong result unless your local timezone is UTC.

Can I just always set is_dst=True?

You can, if you don’t mind getting imprecise results for ambiguous or non-existent times e.g., there is DST transition in the Fall in America/New_York time zone:

>>> from datetime import datetime
>>> import pytz # $ pip install pytz
>>> tz = pytz.timezone('America/New_York')
>>> ambiguous_time = datetime(2015, 11, 1, 1, 30)
>>> time_fmt = '%Y-%m-%d %H:%M:%S%z (%Z)'
>>> tz.localize(ambiguous_time).strftime(time_fmt)
'2015-11-01 01:30:00-0500 (EST)'
>>> tz.localize(ambiguous_time, is_dst=False).strftime(time_fmt) # same
'2015-11-01 01:30:00-0500 (EST)'
>>> tz.localize(ambiguous_time, is_dst=True).strftime(time_fmt) # different
'2015-11-01 01:30:00-0400 (EDT)'
>>> tz.localize(ambiguous_time, is_dst=None).strftime(time_fmt) 
Traceback (most recent call last):
...
pytz.exceptions.AmbiguousTimeError: 2015-11-01 01:30:00

The clocks are turned back at 2a.m. on the first Sunday in November:

is_dst disambiguation flag may have three values:

• False — default, assume the winter time
• True — assume the summer time
• None — raise an exception for ambiguous/non-existent times.

is_dst value is ignored for existing unique local times.

Here’s a plot from PEP 0495 — Local Time Disambiguation that illustrates the DST transition:

The local time repeats itself twice in the fold (summer time — before the fold, winter time — after).

To be able to disambiguate the local time automatically, you need some additional info e.g., if you read a series of local times then it may help if you know that they are sorted: Parsing of Ordered Timestamps in Local Time (to UTC) While Observing Daylight Saving Time.

Method 2

First of all '%s' is not supported on all platforms , its actually working for you because your platform C library’s strftime() function (that is called by Python) supports it. This function is what is causing the issue most probably, I am guessing its not timezone aware , hence when taking difference from epoch time it is using your local timezone, which is most probably EST(?)

Instead of relying on '%s' , which only works in few platforms (linux, I believe) , you should manually subtract the datetime you got from epoch (1970/1/1 00:00:00) to get the actual seconds since epoch . Example –

e = (utc_dt - datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.utc)).total_seconds()

Demo –

>>> (utc_dt - datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.utc)).total_seconds()
1429578727.0

This correctly corresponds to the date-time you get.

Method 3

I don’t exactly know why but you have to remove the timezone info from your utc_dt before using %s to print it.

e = int(utc_dt.replace(tzinfo=None).strftime("%s"))
print(e)
return e

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0