We present a novel approach for zero-shot segmentation of electron microscopy (EM) images, where the masks obtained by the Segment Anything 2 (SAM2) model are clustered in order to retrieve just the objects of interest and filter out other structures in the image. Additionally, we propose a novel clustering algorithm inspired by the agglomerative bottom-up clustering approach that shows promising performance on two EM datasets. The results indicate that by using the proposed approach, we can significantly improve the performance of the vanilla SAM2 model without requiring any additional training data.