Converting time zone pandas dataframe

I have data:

                             Symbol      bid      ask
Timestamp                                            
2014-01-01 21:55:34.378000  EUR/USD  1.37622  1.37693
2014-01-01 21:55:40.410000  EUR/USD  1.37624  1.37698
2014-01-01 21:55:47.210000  EUR/USD  1.37619  1.37696
2014-01-01 21:55:57.963000  EUR/USD  1.37616  1.37696
2014-01-01 21:56:03.117000  EUR/USD  1.37616  1.37694

The timestamp is in GMT. Is there a way to convert that to Eastern?

Note when I do:

data.index

I get output:

<class 'pandas.tseries.index.DatetimeIndex'>
[2014-01-01 21:55:34.378000, ..., 2014-01-01 21:56:03.117000]
Length: 5, Freq: None, Timezone: None

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

Localize the index (using tz_localize) to UTC (to make the Timestamps timezone-aware) and then convert to Eastern (using tz_convert):

import pytz
eastern = pytz.timezone('US/Eastern')
df.index = df.index.tz_localize(pytz.utc).tz_convert(eastern)

For example:

import pandas as pd
import pytz

index = pd.date_range('20140101 21:55', freq='15S', periods=5)
df = pd.DataFrame(1, index=index, columns=['X'])
print(df)
#                      X
# 2014-01-01 21:55:00  1
# 2014-01-01 21:55:15  1
# 2014-01-01 21:55:30  1
# 2014-01-01 21:55:45  1
# 2014-01-01 21:56:00  1

# [5 rows x 1 columns]
print(df.index)
# <class 'pandas.tseries.index.DatetimeIndex'>
# [2014-01-01 21:55:00, ..., 2014-01-01 21:56:00]
# Length: 5, Freq: 15S, Timezone: None

eastern = pytz.timezone('US/Eastern')
df.index = df.index.tz_localize(pytz.utc).tz_convert(eastern)
print(df)
#                            X
# 2014-01-01 16:55:00-05:00  1
# 2014-01-01 16:55:15-05:00  1
# 2014-01-01 16:55:30-05:00  1
# 2014-01-01 16:55:45-05:00  1
# 2014-01-01 16:56:00-05:00  1

# [5 rows x 1 columns]

print(df.index)
# <class 'pandas.tseries.index.DatetimeIndex'>
# [2014-01-01 16:55:00-05:00, ..., 2014-01-01 16:56:00-05:00]
# Length: 5, Freq: 15S, Timezone: US/Eastern

Method 2

The simplest way is to use to_datetime with utc=True:

df = pd.DataFrame({'Symbol': ['EUR/USD'] * 5,
                  'bid': [1.37622, 1.37624, 1.37619, 1.37616, 1.37616],
                  'ask': [1.37693, 1.37698, 1.37696, 1.37696, 1.37694]})

df.index = pd.to_datetime(['2014-01-01 21:55:34.378000',
                          '2014-01-01 21:55:40.410000',
                          '2014-01-01 21:55:47.210000',
                          '2014-01-01 21:55:57.963000',
                          '2014-01-01 21:56:03.117000'],
                           utc=True)

For more flexibility, you can convert timezones with tz_convert(). If your data column/index is not timezone-aware, you will get a warning, and should first make the data timezone-aware with tz_localize.

df = pd.DataFrame({'Symbol': ['EUR/USD'] * 5,
                  'bid': [1.37622, 1.37624, 1.37619, 1.37616, 1.37616],
                  'ask': [1.37693, 1.37698, 1.37696, 1.37696, 1.37694]})

df.index = pd.to_datetime(['2014-01-01 21:55:34.378000',
                          '2014-01-01 21:55:40.410000',
                          '2014-01-01 21:55:47.210000',
                          '2014-01-01 21:55:57.963000',
                          '2014-01-01 21:56:03.117000'])

df.index = df.index.tz_localize('GMT')
df.index = df.index.tz_convert('America/New_York')

This also works similarly for datetime columns, but you need dt after accessing the column:

df['column'] = df['column'].dt.tz_convert('America/New_York')

Method 3

To convert EST time into Asia tz

df.index = data.index.tz_localize('EST')
df.index = data.index.tz_convert('Asia/Kolkata')

Pandas has now inbuilt tz conversion ability.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0