Huge discrepancies in Logistic Regression and SVM using HOG features to identify an Object
I am doing some research on Logistic regression and SVM using different parameters using HOG features. I am facing a bit of problem while understanding each classifier with combination of different parameters and different HOG features.
My findings and confusions are given below,
For Hog: orientations=18, pixelsPerCell=(6,6), cellsPerBlock=(1,1)
Classifier: SVC(C=1000.0, gamma=0.1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=210,Wrong Prediction=6
Classifier: SVC(C=100.0, gamma=0.1, kernel='rbf')
Output: Total dataset=216, Correct prediction=210,Wrong Prediction=6
Classifier: SVC(C=1.0, gamma=0.1, kernel='rbf')
Output: Total dataset=216, Correct prediction=209,Wrong Prediction=7
,
For Hog: orientations=9, pixelsPerCell=(6,6), cellsPerBlock=(1,1)
Classifier: SVC(C=1000.0, gamma=0.1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=211,Wrong Prediction=5
Classifier: SVC(C=100.0, gamma=0.1, kernel='rbf')
Output: Total dataset=216, Correct prediction=211,Wrong Prediction=5
Classifier: SVC(C=1.0, gamma=0.1, kernel='rbf')
Output: Total dataset=216, Correct prediction=210,Wrong Prediction=6
-------------------------------------------------------------------------
For Hog: orientations=9, pixelsPerCell=(9,9), cellsPerBlock=(3,3)
Classifier: SVC(C=1000.0, gamma=1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=100.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=1.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=213,Wrong Prediction=3
For Hog: orientations=9, pixelsPerCell=(6,6), cellsPerBlock=(3,3)
Classifier: SVC(C=1000.0, gamma=1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=100.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=210,Wrong Prediction=6
Classifier: SVC(C=1.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
For Hog: orientations=18, pixelsPerCell=(9,9), cellsPerBlock=(3,3)
Classifier: SVC(C=1000.0, gamma=1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=100.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=1.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=3
For Hog: orientations=18, pixelsPerCell=(6,6), cellsPerBlock=(3,3)
Classifier: SVC(C=1000.0, gamma=1, kernel='rbf'),
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=100.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
Classifier: SVC(C=1.0, gamma=1, kernel='rbf')
Output: Total dataset=216, Correct prediction=212,Wrong Prediction=4
For the above results we can conclude that "C" of SVM and orientation of HOG doesn't matter much for the combinations, so we do not have the issue of highbias or variance and bin size. What matters is the value of gamma and combination of HOG with "cellsPerBlock".
From my analysis, I found
1. Using "rbf" SVM with HOG "cellsPerBlock=(1,1)" doesn't give proper outcome.
2. The "rbf" SVM with HOG "cellsPerBlock=(1,1)" works best with gamma=0.1
3. The "rbf" SVM with HOG "cellsPerBlock=(3,3)" and different combination of "pixelspercells" works best with gamma=1
I've also tried pixelspercell = (3,3) but all the different combinations of "rbf" kernels fails to classify properly.
"" The problem I face is understanding this discrepancy""
My Confusion even eggravates when I use Logistic regression for the Combinations of HOG surprisingly Regularized Logistic Regression works better with cellsPerBlock=(1,1)
Below are my findings with 2nd order Regularized Logistic Regression: I used 0.9 as my threshold level, i.e below 0.9 would be classifies as 0 and above 0.9 would be classified as 1
Q: Why did I put 0.9 as threshold-
Sol: Because the test data that had the class(labels)=1, 99% of them output probability 0.9
However the test data that had the class(labels)=0, many instances outputprobability 0.5
So inorder to filter out many test data instances that gave probability 0.5 A threshold of 0.9 was used.
1. orientations=9,
pixelsPerCell=(6,6)
cellsPerBlock=(3,3)
Output: Total dataset=216, Correct prediction=187, Wrong Prediction=29
2. orientations=9,
pixelsPerCell=(9,9)
cellsPerBlock=(3,3)
Output: Total dataset=216, Correct prediction=202, Wrong Prediction=14
3. orientations=9,
pixelsPerCell=(9,9)
cellsPerBlock=(1,1)
Output: Total dataset=216, Correct prediction=211, Wrong Prediction=5
4. orientations=9,
pixelsPerCell=(6,6)
cellsPerBlock=(1,1)
Output: Total dataset=216, Correct prediction=206, Wrong Prediction=10
5. orientations=9,
pixelsPerCell=(3,3)
cellsPerBlock=(1,1)
Output: Total dataset=216, Correct prediction=203, Wrong Prediction=13
Topic hog logistic-regression svm r machine-learning
Category Data Science