EV detection on Melbourne House Data | by Kiran J Chemmanatte | Apr, 2024

The EV or Electrical Automobile craze has his Australia with 8.4% of all new autos being supplied in Australia being an EV, A 120% soar from 2022. Although this modification is unbelievable for the enviorment, it props up potential challenges for power suppliers. EV’s dissipate a considerable amount {of electrical} energy to value which is totally top quality when it is just one . Nonetheless, if a block of a suburb has many EV clients and if all of them come once more from work at associated cases and price up their EV’s. It locations the power grid on the place under an unlimited load, doubtlessly even overwhelming the system.

Subsequently being able detect, which properties embrace EVs and determining such clusters of properties in a block is of great significance. That’s the draw back we attempt to resolve at current with ML.

Okay first lets uncover our dataset. We’ve received the power-consumption data of 88 properties in Melbourne, taken at a bihourly charge for 46 days, along with the date , id of the house and finally the label of whether or not or not the house has an EV or not. The csv file along with the codes could also be found proper right here .

Lets uncover the dataset with pandas

#importing the knowledge and viewing it 
df = pd.read_csv("../data/EV_data.csv")
ptdf.head()

As on a regular basis the first step is data cleaning

Step1.1 look at for missing values/ null values

The following code counts the number of null values in each column and exhibits it in a desk

df.isna().sum()

We’re good the dataset incorporates no null values, spectacular to say the least.

Step 1.2 Take a look at if all column datatypes are acceptable.

We observe a difficulty , the read_date column won’t be represented by a date-time format, subsequently to place it to make use of we’ve received to transform the knowledge to a usable sort.

As we see the knowledge in column follows a pattern month/day/12 months so pandas to datatime funtion ought to help with reworking the knowledge.

df['read_date'] = pd.to_datetime(df['read_date'], format="%m/%d/%Y")

Sadly the date column has some entries which moreover incorporates time stamps ‘0:00’ nonetheless that data is redundant as a result of the columns give the time data . So we take out these values after which apply data transformation .

df['read_date'] = df['read_date'].str.substitute(' 0:00', '')
df['read_date'] = pd.to_datetime(df['read_date'], format="%m/%d/%Y")

Certain, success we now have the right datatypes.

Step 2 : Attribute engineering

An important degree to note is that categorical data or string data can’t be utilized by many classification fashions resembling Logistic regression, KNN and Random Forests , subsequently we’ve received to utilize clever methods resembling encoding or create custom-made numeric choices to make the knowledge acceptable for modeling.

Proper right here the read_date won’t be in a numeric format, we’re in a position to extract choices resembling month, day of month, day of week and make them into new choices to utilize them in modelling.

df['day_of_week_num'] = df['read_date'].dt.day_of_week
df['day_of_month'] = df['read_date'].dt.day
df['month'] = df['read_date'].dt.month
df.head()

Very Important !!!!

When you could have noticed , we are able to’t cope with this dataset like a typical dataset, we attempt to foretell whether or not or not each id (dwelling id) has an EV or not. Nonetheless, each id has 46 rows correspoinding to its data. Due to this fact as soon as we stock out the train-test reduce up we’ve received to ensure that rows of id which will be throughout the examine data aren’t throughout the put together data. ie, if dwelling id 50 is in examine data set , no row of id 50 should be throughout the put together dataset.

To carry out this job we reduce up the dataset based mostly totally on their distinctive dwelling ids and populate the teaching dataset and testing dataset based mostly totally on that intial reduce up.

from sklearn.model_selection import train_test_split
train_ids, test_ids = train_test_split(unique_ids, test_size=0.2, random_state=42)
train_data = df.loc[df['id'].isin(train_ids)]
test_data = df.loc[df['id'].isin(test_ids)]

We’ll use Xgboost, KNN , Logistic regression and RF(random forest) as fashions and use 5-fold cross validation.

from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
import xgboost as xgb
from sklearn.model_selection import KFold, train_test_split
from sklearn.metrics import accuracy_score

The code needed to teach and examine the fashions is given underneath.

# Prepare the KFold cross-validation
kfold = KFold(n_splits=5, shuffle=True, random_state=42)
# Lists to retailer the scores from each fold
train_scores_lr = []
val_scores_lr = []
train_scores_rf = []
val_scores_rf = []
train_scores_knn = []
val_scores_knn = []
train_scores_xgb = []
val_scores_xgb = []
# Iterate over the folds
for train_idx, val_idx in kfold.reduce up(X_train, y_train):
# Break up the teaching data into put together and validation items
X_train_fold, X_val_fold = X_train.iloc[train_idx], X_train.iloc[val_idx]
y_train_fold, y_val_fold = y_train.iloc[train_idx], y_train.iloc[val_idx]
# Create the fashions
lr_model = LogisticRegression(random_state=42)
rf_model = RandomForestClassifier(random_state=42)
knn_model = KNeighborsClassifier()
xgb_model = xgb.XGBClassifier(objective="binary:logistic", random_state=42)
# Put together the fashions on the teaching fold
lr_model.match(X_train_fold, y_train_fold)
rf_model.match(X_train_fold, y_train_fold)
knn_model.match(X_train_fold, y_train_fold)
xgb_model.match(X_train_fold, y_train_fold)
# Take into account the fashions on the validation fold
train_score_lr = accuracy_score(y_train_fold, lr_model.predict(X_train_fold))
val_score_lr = accuracy_score(y_val_fold, lr_model.predict(X_val_fold))
train_score_rf = accuracy_score(y_train_fold, rf_model.predict(X_train_fold))
val_score_rf = accuracy_score(y_val_fold, rf_model.predict(X_val_fold))
train_score_knn = accuracy_score(y_train_fold, knn_model.predict(X_train_fold))
val_score_knn = accuracy_score(y_val_fold, knn_model.predict(X_val_fold))
train_score_xgb = accuracy_score(y_train_fold, xgb_model.predict(X_train_fold))
val_score_xgb = accuracy_score(y_val_fold, xgb_model.predict(X_val_fold))
# Append the scores to the lists
train_scores_lr.append(train_score_lr)
val_scores_lr.append(val_score_lr)
train_scores_rf.append(train_score_rf)
val_scores_rf.append(val_score_rf)
train_scores_knn.append(train_score_knn)
val_scores_knn.append(val_score_knn)
train_scores_xgb.append(train_score_xgb)
val_scores_xgb.append(val_score_xgb)
print(f'Fold {len(train_scores_lr)}:')
print(f'Logistic Regression: Put together Accuracy = {train_score_lr:.4f}, Validation Accuracy = {val_score_lr:.4f}')
print(f'Random Forest: Put together Accuracy = {train_score_rf:.4f}, Validation Accuracy = {val_score_rf:.4f}')
print(f'KNN: Put together Accuracy = {train_score_knn:.4f}, Validation Accuracy = {val_score_knn:.4f}')
print(f'XGBoost: Put together Accuracy = {train_score_xgb:.4f}, Validation Accuracy = {val_score_xgb:.4f}')
# Print the suggest scores
print('nMean Scores:')
print(f'Logistic Regression: Suggest Put together Accuracy = {sum(train_scores_lr) / len(train_scores_lr):.4f}, Suggest Validation Accuracy = {sum(val_scores_lr) / len(val_scores_lr):.4f}')
print(f'Random Forest: Suggest Put together Accuracy = {sum(train_scores_rf) / len(train_scores_rf):.4f}, Suggest Validation Accuracy = {sum(val_scores_rf) / len(val_scores_rf):.4f}')
print(f'KNN: Suggest Put together Accuracy = {sum(train_scores_knn) / len(train_scores_knn):.4f}, Suggest Validation Accuracy = {sum(val_scores_knn) / len(val_scores_knn):.4f}')
print(f'XGBoost: Suggest Put together Accuracy = {sum(train_scores_xgb) / len(train_scores_xgb):.4f}, Suggest Validation Accuracy = {sum(val_scores_xgb) / len(val_scores_xgb):.4f}')
# Put together the final word fashions on all of the teaching set
final_lr_model = LogisticRegression(random_state=42)
final_rf_model = RandomForestClassifier(random_state=42)
final_knn_model = KNeighborsClassifier()
final_xgb_model = xgb.XGBClassifier(objective="binary:logistic", random_state=42)
final_lr_model.match(X_train, y_train)
final_rf_model.match(X_train, y_train)
final_knn_model.match(X_train, y_train)
final_xgb_model.match(X_train, y_train)
# Take into account the final word fashions on the examine set
test_accuracy_lr = accuracy_score(y_test, final_lr_model.predict(X_test))
test_accuracy_rf = accuracy_score(y_test, final_rf_model.predict(X_test))
test_accuracy_knn = accuracy_score(y_test, final_knn_model.predict(X_test))
test_accuracy_xgb = accuracy_score(y_test, final_xgb_model.predict(X_test))
print('nTest Accuracies:')
print(f'Logistic Regression: {test_accuracy_lr:.4f}')
print(f'Random Forest: {test_accuracy_rf:.4f}')
print(f'KNN: {test_accuracy_knn:.4f}')
print(f'XGBoost: {test_accuracy_xgb:.4f}')

Lets see the examine outcomes , so Xgboost has carried out the right adopted by RF

Mainly the right model is able to appropriately classify the knowledge with an accuracy of 83% unbelievable. Now lets see the attribute significance to see which choices contributed the model offer you predictions.

# Get the attribute importances
feature_importances = final_xgb_model.get_booster().get_score(importance_type="weight")
# Convert the attribute importances to a DataFrame
feature_importances = pd.DataFrame(feature_importances.objects(), columns=['feature', 'importance'])
# Sort the DataFrame by significance in descending order
feature_importances = feature_importances.sort_values('significance', ascending=False)
# Print the best 10 most important choices
print("Prime 10 Important Choices:")
print(feature_importances.head(10))
# Plot the best 20 most important choices
plt.decide(figsize=(10, 6))
feature_importances.head(20).plot.bar(x='attribute', y='significance')
plt.title('Attribute Importances')
plt.xlabel('Attribute')
plt.ylabel('Significance')
plt.xticks(rotation=90)
plt.tight_layout()
plt.current()

Okay so the morning hours gave the model basically probably the most useful data and the choices that we engineered moreover helped barely , based mostly totally on this we’re in a position to further refine the model or engineer totally different choices nonetheless that’s for an extra time.

Thanks for taking your time to be taught my weblog, hope you found value sort it.