Assignment 2

Before working on this assignment please read these instructions fully. In the submission area, you will notice that you can click the link to Preview the Grading for each step of the assignment. This is the criteria that will be used for peer grading. Please familiarize yourself with the criteria before beginning the assignment.

An NOAA dataset has been stored in the file data/C2A2_data/BinnedCsvs_d18/93c648398ff85fad51308f4ff8d11c2e8d8e66392462ffe79f3fb628.csv. The data for this assignment comes from a subset of The National Centers for Environmental Information (NCEI) Daily Global Historical Climatology Network (GHCN-Daily). The GHCN-Daily is comprised of daily climate records from thousands of land surface stations across the globe.

Each row in the assignment datafile corresponds to a single observation.

The following variables are provided to you:

  • id : station identification code
  • date : date in YYYY-MM-DD format (e.g. 2012-01-24 = January 24, 2012)
  • element : indicator of element type
    • TMAX : Maximum temperature (tenths of degrees C)
    • TMIN : Minimum temperature (tenths of degrees C)
  • value : data value for element (tenths of degrees C)

For this assignment, you must:

  1. Read the documentation and familiarize yourself with the dataset, then write some python code which returns a line graph of the record high and record low temperatures by day of the year over the period 2005-2014. The area between the record high and record low temperatures for each day should be shaded.
  2. Overlay a scatter of the 2015 data for any points (highs and lows) for which the ten year record (2005-2014) record high or record low was broken in 2015.
  3. Watch out for leap days (i.e. February 29th), it is reasonable to remove these points from the dataset for the purpose of this visualization.
  4. Make the visual nice! Leverage principles from the first module in this course when developing your solution. Consider issues such as legends, labels, and chart junk.

The data you have been given is near Jeju City, Jeju-do, Republic of Korea, and the stations the data comes from are shown on the map below.

Packages

In [1]:
import matplotlib.pyplot as plt
import mplleaflet
import pandas as pd
import numpy as np

Given Code

In [2]:
def leaflet_plot_stations(binsize, hashid):

    df = pd.read_csv('data/C2A2_data/BinSize_d{}.csv'.format(binsize))
    
    station_locations_by_hash = df[df['hash'] == hashid]
    lons = station_locations_by_hash['LONGITUDE'].tolist()
    lats = station_locations_by_hash['LATITUDE'].tolist()

    plt.figure(figsize=(8,8))

    plt.scatter(lons, lats, c='r', alpha=0.7, s=200)

    return mplleaflet.display()

leaflet_plot_stations(18,'93c648398ff85fad51308f4ff8d11c2e8d8e66392462ffe79f3fb628')
Out[2]:

Data Manipulation

In [3]:
import warnings
warnings.filterwarnings('ignore')

#Reading the raw data.
weather = pd.read_csv('data/C2A2_data/BinnedCsvs_d18/93c648398ff85fad51308f4ff8d11c2e8d8e66392462ffe79f3fb628.csv')

#Changing temperature from tenths to whole degrees and removing the two leap days.
weather['Data_Value']= weather['Data_Value']/10
weather = weather[weather['Date'] != '2008-02-29']
weather = weather[weather['Date'] != '2012-02-29']

#Extracting all data from 2005 - 2014.  Then removing the year, changing YYYY-MM-DD
#to MM-DD.  After, a new data frame is created with the date, minimum, and maximum
#temperature for that date.
weather_2005_2014 = weather[weather['Date'] < '2015-01-01']
weather_2005_2014['Date'] = weather_2005_2014['Date'].str[5:]
temps = (weather_2005_2014.groupby(['Date'])['Data_Value'].min()).to_frame(name = 'DECADE_TMIN')
temps['DECADE_TMAX'] = weather_2005_2014.groupby(['Date'])['Data_Value'].max()

#Extracting all data from 2015.  Then removing the year, as before.  The 2015
#minimum and maximum temperatures for each date are appended to the previous data
#frame.  The index is reset.
weather_2015 = weather[weather['Date'] >= '2015-01-01']
weather_2015['Date'] = weather_2015['Date'].str[5:]
temps['2015_TMIN'] = weather_2015.groupby(['Date'])['Data_Value'].min()
temps['2015_TMAX'] = weather_2015.groupby(['Date'])['Data_Value'].max()
temps = temps.reset_index()
temps.index.name = 'Day'
temps = temps.reset_index()

#Two warnings will be output because 'Date' is being modified on two slices of
#weather by .str[5:].  This is not an error.

#Helper function for next four dataframes.  It creates a two column dataframe
#with a separate index.  The two columns are day of the year and corresponding
#temperature for that characterizes that dataframe.
def df_day_temp(new_df, old_df, temp_col):
    new_df = old_df[['Day', temp_col]].reset_index(drop=True)
    new_df.rename(columns={temp_col: 'Temperature'}, inplace = True)
    return new_df

#Four, simplified, two column data frames are sliced from temps.  These will be
#used for plotting.

#Minimum temperature for each day of the year from 2005 - 2014.
min_decade = pd.DataFrame()
min_decade = df_day_temp(min_decade, temps, 'DECADE_TMIN')

#Maximum temperature for each day of the year from 2005 - 2014.
max_decade = pd.DataFrame()
max_decade = df_day_temp(max_decade, temps, 'DECADE_TMAX')

#Day and temperature for 2015 that was less than that of previous decade.
min_2015 = pd.DataFrame()
min_2015 = df_day_temp(min_2015, temps[temps['DECADE_TMIN'] > temps['2015_TMIN']], '2015_TMIN')

#Day and temperature for 2015 that was greater than that of previous decade.
max_2015 = pd.DataFrame()
max_2015 = df_day_temp(max_2015, temps[temps['DECADE_TMAX'] < temps['2015_TMAX']], '2015_TMAX')

Plot

In [4]:
%matplotlib notebook
plt.figure(figsize=(10,6.5))

#Axis set for days of year and 5° above and below the maximum and minimum.
ax = plt.gca()
ax.axis([0, 365, min(min_decade['Temperature']) - 5, max(max_decade['Temperature']) + 5
        ])

#Ticks set for first day of each month.
plt.xticks([1, 32, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335],
           ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',
            'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'],
           rotation = 75)

#Labels
plt.xlabel('Day', fontweight = "bold")
plt.ylabel('Temperature (°C)', fontweight = "bold")
plt.title('2015 Extreme Temperatures Compared to 2005-2014\nJeju-do, South Korea', fontweight = "bold")

#Maximum temperature for each day of the year from 2005 - 2014.
plt.plot(max_decade['Day'],
         max_decade['Temperature'],
         c = 'lightcoral',
         zorder = 1)

#Minimum temperature for each day of the year from 2005 - 2014.
plt.plot(min_decade['Day'],
         min_decade['Temperature'],
         c = 'paleturquoise',
         zorder = 1)

#Day and temperature for 2015 that was greater than that of previous decade.
plt.scatter(max_2015['Day'],
            max_2015['Temperature'], s = 100, c = 'red', zorder = 2)

#Day and temperature for 2015 that was less than that of previous decade.
plt.scatter(min_2015['Day'],
            min_2015['Temperature'], s = 100, c = 'blue', zorder = 2)

#Shading.
plt.gca().fill_between(min_decade['Day'], min_decade['Temperature'], max_decade['Temperature'], 
                       facecolor = 'grey', alpha = 0.2)

#Legend.
plt.legend(['Maximum 2005 - 2014',
            'Minimum 2005 - 2014',
            'Extreme 2015',
            'Extreme 2015',
            'All Other Temperatures, 2005 - 2015'],
           loc = 8,
           title = 'Temperatures by Day');