In recent years, region-based active contour models have gained great popularity in solving image segmentation problem. Those models usually share two assumptions regarding the image pixel properties: 1) within each region/ object, the intensity values conform to a Gaussian distribution; 2) the "global mean" (average intensity value) for different regions are distinct, therefore can be used in discriminating pixels. These two assumptions are often violated in reality, which results in segmentation leakage or misclassification. In this paper, we propose a robust segmentation framework that overcomes the above mentioned drawback existing in most region-based active contour models. Our framework consists of two components: 1) instead of using a global average intensity value (mean) to represent certain region, we use local medians as the region representative measure to better characterize the local property of the image; 2) median and sum of absolute values (L1 norm) is used to formulate the energy minimization functional for better handling intensity variations and outliers. Experiments are conducted on several real images, and we compare our solution with a popular region-based model to show the improvements.