Adapt Machine Learning Algorithm for overridden decisions
We have insurance data of over 10 years. There are underwriting rules for the data, which result in two possible outcomes: Approve or Reject.
We want to have a Machine Learning Algorithm to learn these rules and predict the outcome for the future cases, which is all fine. BUT, if the socio-economic condition changes, the underwriter will override the ML decision manually. It is expected for the system to adapt accordingly and behave in that fashion for the upcoming applications.
Is there any possible way(s)?
See also questions close to this topic
RandomForestClassifier instance not fitted yet. Call 'fit' with appropriate arguments before using this method
I am trying to train a decision tree model, save it, and then reload it when I need it later. However, I keep getting the following error:
This DecisionTreeClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this method.
Here is my code:
X_train, X_test, y_train, y_test = train_test_split(data, label, test_size=0.20, random_state=4) names = ["Decision Tree", "Random Forest", "Neural Net"] classifiers = [ DecisionTreeClassifier(), RandomForestClassifier(), MLPClassifier() ] score = 0 for name, clf in zip(names, classifiers): if name == "Decision Tree": clf = DecisionTreeClassifier(random_state=0) grid_search = GridSearchCV(clf, param_grid=param_grid_DT) grid_search.fit(X_train, y_train_TF) if grid_search.best_score_ > score: score = grid_search.best_score_ best_clf = clf elif name == "Random Forest": clf = RandomForestClassifier(random_state=0) grid_search = GridSearchCV(clf, param_grid_RF) grid_search.fit(X_train, y_train_TF) if grid_search.best_score_ > score: score = grid_search.best_score_ best_clf = clf elif name == "Neural Net": clf = MLPClassifier() clf.fit(X_train, y_train_TF) y_pred = clf.predict(X_test) current_score = accuracy_score(y_test_TF, y_pred) if current_score > score: score = current_score best_clf = clf pkl_filename = "pickle_model.pkl" with open(pkl_filename, 'wb') as file: pickle.dump(best_clf, file) from sklearn.externals import joblib # Save to file in the current working directory joblib_file = "joblib_model.pkl" joblib.dump(best_clf, joblib_file) print("best classifier: ", best_clf, " Accuracy= ", score)
Here is how I load the model and test it:
#First method with open(pkl_filename, 'rb') as h: loaded_model = pickle.load(h) #Second method joblib_model = joblib.load(joblib_file)
As you can see, I have tried two ways of saving it but none has worked.
Here is how I tested:
You can clearly see that the models are actually fitted and if I try with any other models such as SVM, or Logistic regression the method works just fine.
Handling outlier for e-commerce data
I have read about deleting the data points that is outlier, I also know about winsorizing the data also to change the upper and lower inner fence.
Is there any better solution for handling the outlier in e-commerce dataset? Thank you :)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() when converting a map object to list
I've title_result which is a list of 1000 movie titles.
i've a function whivh return similar movies based on a model. The function looks like this:
def get_recommendations(title): idx = indices[title] sim_scores = list(enumerate(cosine_sim[idx])) sim_scores = sorted(sim_scores, key=lambda x: x, reverse=True) sim_scores = sim_scores[1:31] movie_indices = [i for i in sim_scores] return title, str(titles.iloc[movie_indices].reset_index(drop=True).tolist())
The output of function will be as folllows when 'Twilight Saga: Breaking Dawn - 1' is passed as input parameter.
('Twilight Saga: Breaking Dawn - 1', "['Twilight Saga: Breaking Dawn - 2', 'Kedi Billa Killadi Ranga', 'Anjali', 'Half Girlfriend']")
I've used a map function to get result for rest of the titles.
result = list(map(get_recommendations,title_result)) result_df = pd.DataFrame(result,columns=['title','similar_movies'])
When I pass list range title_result[:20], I will get the output perfectly, but when when I pass whole list or list range from [25:30], I'm getting
`ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()`.
I think there is some exception with certain values when converting map object to list, which I'm unable to solve.
simple machine learning library with text pattern recognition
I was looking online and everything looked really complicated so I wanted to know if anyone knew of a python library where I could feed it a bunch of inputs and outputs and from those it would learn what the outputs it should give. So pretty much it can learn from having conversation or watching others. just a simple learning library. Hope that made sense!
AttributeError: module 'tensorflow.python.training.checkpointable' has no attribute 'CheckpointableBase'
I've been working on learning artificial intelligence and how to code with Python.I was working on a project and I decided to update some packages of Python which were not new to work on then something happened and I can't compile my codes.I deleted Anaconda3 and set it up again but not worked. I've been seeing this problem which I wrote as a topic.If someone helps me,I would be appriciated to get some help.
>>> import tensorflow as tf File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\__init__.py", line 24, in <module> from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\__init__.py", line 63, in <module> from tensorflow.python.framework.framework_lib import * # pylint: disable=redefined-builtin File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\framework\framework_lib.py", line 104, in <module> from tensorflow.python.framework.importer import import_graph_def File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\framework\importer.py", line 32, in <module> from tensorflow.python.framework import function File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\framework\function.py", line 36, in <module> from tensorflow.python.ops import resource_variable_ops File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 35, in <module> from tensorflow.python.ops import variables File "C:\Users\AliGalip\Anaconda3Yeni\lib\site-packages\tensorflow\python\ops\variables.py", line 40, in <module> class Variable(checkpointable.CheckpointableBase): AttributeError: module 'tensorflow.python.training.checkpointable' has no attribute 'CheckpointableBase'
Fake news detection using AI
How can we detect fake news spread over the social media by using artificial intelligence tools and machine learning.what are the algorithms used to detect fake things.how the artificial intelligence model can be done