Calculating the F score of Object Detection of Mask RCNN

I am using Detectron2 Mask RCNN for an object detection problem. The images consist of cells that are very close to each other. I can not use mAP as a performance measure since the annotations are a bit off from the original location and the prediction is actually more accurate and when I use mAP it will give bad results.

Generally, each cell is 30 pixels apart and if the predicted and the actual are less then 30 pixels apart I count that prediction as a TP. I compare each prediction with each actual cell and if an actual cell is left at the end and no predicted is 30 pixels near it I add that to the FP.

This way I have TP, FP and the length of the actual cell is my FN. I calculate the Precision and the Recall and with these, I can calculate the F-Score.

Problem: When the threshold is small for making predictions I get an F-score greater than 1.

Here is my code

TP=0
FN=0
FP=0

for d in tqdm(dataset_dicts):

    bboxs = []
    bbox_center_x = []
    bbox_center_y = []


    predicted_centers_x = []
    predicted_centers_y = []

    path = d["file_name"].replace('\\','/')
    im = cv2.imread(path)

    outputs = predictor(im) #going over image by image to find predictions

    i = d['annotations']
    for i in range(len(d['annotations'])):
        bboxs.append(d['annotations'][i]['bbox']) #bboxs have the 

    bboxs = np.array(bboxs)
    predicted_bboxs = outputs["instances"].pred_boxes.to("cpu").tensor.numpy()

    for bbox in bboxs:
        bbox_center_x.append((bbox[0]+bbox[0]+bbox[2])/2)
        bbox_center_y.append((bbox[1]+bbox[1]+bbox[3])/2)

    for predicted_bbox in predicted_bboxs:
        predicted_centers_x.append((predicted_bbox[0]+predicted_bbox[2])/2)
        predicted_centers_y.append((predicted_bbox[1]+predicted_bbox[3])/2)



    num_tumor = len(bbox_center_x)
    pred_num_tumor = len(predicted_centers_x)
    if num_tumor  pred_num_tumor:
        FN += num_tumor - pred_num_tumor

    temp_bbox_x = bbox_center_x
    temp_bbox_y = bbox_center_y

    for i in range(len(predicted_centers_x)):
        for j in range(len(temp_bbox_x)):
            if (math.sqrt( (temp_bbox_x[j] - predicted_centers_x[i])**2 + 
                (temp_bbox_y[j] - predicted_centers_y[i])**2 ) ) = 30:
                TP += 1

                break
            if j == ( len(temp_bbox_x)-1 ):
                FP += 1




Precision = (TP/(TP+FP))
Recall = (TP/(TP+FN))

f_score = 2*(Recall * Precision) / (Recall + Precision)
print(f_score)