2017 IEEE 8th Control and System Graduate Research Colloquium (ICSGRC 2017), 4 - 5 August 2017, Shah Alam, Malaysia Shape Characterization of Land Covers using SuperResolution Mapping Anuar M. Muad Department of Electrical, Electronic and Systems Engineering Faculty of Engineering and Built Environment Universiti Kebangsaan Malaysia 43600 UKM Bangi, Selangor, Malaysia email@example.com system , geostatistical , and spatial regularization model . Abstract—This paper presents a representation of land cover from a popular low spatial resolution of remote sensing image, MODIS 250 m. The spatial resolution of the MODIS image is enhanced using super-resolution mapping to a resolution that is equal to resolution of Landsat ETM+, which is 30 m. Two superresolution mapping techniques, Hopfield neural network and pixel swapping are used to represent the land covers as patch objects. Parameters for both techniques are varies to investigate their impact towards the characterization of the object in a single MODIS image and also in a time-series MODIS images. This paper investigates the impact of shape characterization from land cover represented by SRM. Here, the HNN and PS were used on a single low spatial resolution of MODIS 250 m and a series of MODIS images. II. HOPFIELD NEURAL NETWORK Hopfield neural network (HNN) is an optimization technique represented as an energy function. The energy function that represents the problem of SRM comprised of two parts: a goal function and a constraint function , as defined as E wGG x wA A x where G(.) is a goal function, A(.) is an area constraint function, wG is a weight coefficient for the goal function and wA is a weight coefficient for the constraint function. The goal function emphasises spatial correlation between sub-pixels while the constraint function preserves class proportion in a pixel of a soft classified image . The network comprises a set of neurons in a two dimensional arrangement in which neuron (i,j) corresponds to sub-pixel (i,j). In this paper, HNN with wA > wG, is referred as HNN(A), while when the weights are equal, wA = wG, the HNN is referred as HNN(E). To exploit the advantages of the HNN(A) and HNN(E), a technique known as HNN2  is used. Keywords—Remote sensing; Hopfield neural network; pixel swapping; object based remote sensing;MODIS; Landsat I. INTRODUCTION Land cover is one of the most fundamental variable for range of studies about the surface of the earth. Land cover can be treated as an object in order to study their size, shape, boundary, and orientation. Accurate object characterization is essential to estimate the impact of land cover and land uses in one region, such as biodiversity, urbanization, vegetation, and many others. Remote sensing has been considered as the most practical method to acquire information about land cover [1-3]. Although remote sensing is widely used as a source of land cover information there are many factors that limit the accuracy of derived land cover information  especially the spatial resolution of the data. Nevertheless, increasing the spatial resolution of the imaging sensor would increases the cost of the device. This factor hugely affects the price of purchasing high spatial resolution imagery from remote sensing agencies. On contrary, low spatial resolution imagery is cheaper than high spatial resolution imagery capturing on the same area . However, a major drawback of the low spatial resolution imagery is that the information is less detail than the high spatial resolution imagery . Traditional hard classifiers  are unable to map the land cover at a subpixel scale of the low spatial resolution image. III. PIXEL SWAPPING Pixel swapping (PS) is a technique that calculates the attractiveness of each sub-pixel to a particular class of land cover in a low spatial resolution image . This technique starts after an image has been soft classified. Pixel swapping uses an attractive function to measure an attractiveness level of a sub-pixel i related to its neighbours j = 1,…,J as given in Equation 2 that measures the summation of weighed distance between a sub-pixel of interest and its neighbours. To enhance the value of low spatial resolution, superresolution mapping (SRM) technique is used. There are many variation of SRM techniques, such as Hopfield neural network (HNN) [8-11], pixel swapping (PS) [12,13], back-propagation neural network , Markov random field , multi agent 978-1-5386-0380-2/17/$31.00 ©2017 IEEE J Fi x i ij j 1 x j where z(xj) is the binary class of the jth sub-pixel at location xj , and ij is a non-linear distance weighted function. In this paper, 57 2017 IEEE 8th Control and System Graduate Research Colloquium (ICSGRC 2017), 4 - 5 August 2017, Shah Alam, Malaysia PS with one layer of neighbour is referred as PS(1), while for five layers of neighbour is referred as PS(5). IV. IMPLEMENTATION The study area covers approximately a 26 km × 28 km area located in Ontario province, Canada. It is situated between latitudes 54o26’24”N and 54o12’10”N and between longitudes 85o26’15”W and 85o01’22”W. The area contains huge number of lakes of varying size and shape. Two sets of image data were used. First, a time series of low spatial resolution imagery of MODIS 250 m was acquired between 13 June 2002 and 19 August 2002. Only near infrared band (0.841-0.876 µm) was used because land and water are highly contrast in this spectrum. There were 14 MODIS images in the series. The MODIS images were provided by the USGS Land Processes Distributed Active Archive Center (LP DAAC) website (https://lpdaac.usgs.gov/). An example of a MODIS image acquired on 5 July 2002 is shown in Fig. 1a. The MODIS images were fused using a phase correlation technique [21,22]. Second, a Landsat ETM+ of 30 m of the region acquired on 10 July 2002 was used to provide ground data as shown in Fig. 1b. Only the near infrared band image (0.750-0.900 µm) was used. The Landsat image was provided by the U.S Geological Survey (USGS) Global Visualization Viewer website (http://glovis.usgs.gov/). The date of the Landsat image lies in the middle of the MODIS images. The MODIS image acquired on 5 July 2002 was used as a reference image for time series image because its date of acquisition is the closest to the date of Landsat. Fig. 2 shows the position of the reference and ground data images. Fig. 2. Temporal coverage of a time-series daily MODIS 250 m. A MODIS image acquired on 5 July 2002 was used as a reference image for a time series image registration. Landsat image acquired on 10 July 2002 was used as a ground data as a means to set the orientation of the MODIS image correctly V. RESULTS The shape of objects can be characterized by area, perimeter, compactness, length and orientation. The compactness of an object is calculated by = 4A/P2, where A is area and P is perimeter. The index ranges from 0 to 1, where values approaching 1 indicate that the shape is circular while values approaching 0 are for linear shape. The evaluation of the positional accuracy compares the boundary of the represented objects with the boundary of the corresponding objects from the ground data. To generate boundary position, the boundary of the objects was vectorised. The vector difference of the boundary positions between the represented objects and the ground data was expressed in root mean squared error (RMSE), which provides a statistical measure of the positional accuracy of objects. The spatial resolution of the MODIS images was enhanced from 250 m to 30 m, which is equal to the spatial resolution of the Landsat image. The enhancement process decomposed a pixel into a number of sub-pixels. In this work, there were 872×936 sub-pixels that eventually become the neurons for the HNN that was arranged in a layer of two dimensional. The HNNs were calculated numerically for 10,000 iterations For the PS technique, these sub-pixels were swapped to produce a representation of land cover. (a) In this work, the lakes were treated as objects. Fig. 3 shows 30 lakes of varying sizes and shapes that were selected and annotated from the ground data. Corresponding lakes in the images of land cover mapping derived from different techniques were compared with the ground data. (b) Fig. 1. Datasets (a) one of the MODIS 250 m near IR images acquired on 5 July 2002 and (b) Landsat ETM+ 30 m near IR image taken on 10 July 2002 that was used as ground data. Fig. 3. Selected lakes used for object based analysis 58 2017 IEEE 8th Control and System Graduate Research Colloquium (ICSGRC 2017), 4 - 5 August 2017, Shah Alam, Malaysia (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) Fig. 4. Boundary fitting for different techniques on Lake 25. The red line indicates the boundary of the lake from the ground data image, and the blue line indicates the boundary of the represented lake. (a) Ground data.(b) Hard classifier. (c) PS(1) on 1 image. (d) PS(5) on 1 image. (e) HNN(E) on 1 image. (f) HNN(A) on 1 image. (g) HNN2 on 1 image. (h) PS(1) on fused image. (i) PS(1) on fused image. (j) HNN(E) on fused image. (k) HNN(A) on fused image. (l) HNN2 on fused image. Table 1. Average difference of area (km2) for 30 lakes. Technique Single image Fused image HC PS (1) PS (5) HNN (E) HNN (A) HNN2 0.1696 0.1471 0.2009 0.1992 0.2006 0.1991 0.1336 0.1425 0.1313 0.1973 0.1290 applied on the fused image produced 0.7218 m error. The HNN(A) that was applied on a fused image produced the highest error with 1.4164 m. For both cases of the single MODIS image and the fused image, the HNN(A) tended to produce high error because the representation of the boundary of the lakes appear to be irregular, serrated and not smooth. Table 3 provides an average of the difference object compactness compared to the ground data. For the compactness measurement, the PS(5) applied on the fused image produce the lowest error with the average error was 0.1858, while the HNN(E) applied on the fused image produced 0.1932 error and the HNN2 applied on the fused image produced 0.1961 error. The PS(1) applied on a fused image produced the highest error with the average error was 0.2460. This scenario suggested that in comparison with other techniques, the shape of the lakes represented by the PS(5) and HNN(E) produced the closest appearance to the corresponding lakes in the ground data. Table 2. Average difference of perimeter (m) for 30 lakes. Technique Single image Fused image HC PS (1) PS (5) HNN (E) HNN (A) HNN2 1.1223 0.9910 1.0981 1.1394 1.3259 1.1394 1.1766 0.6732 0.7354 1.4164 0.7218 Table 3. Average difference of compactness for 30 lakes. Technique Single image Fused image HC PS (1) PS (5) HNN (E) HNN (A) HNN2 0.2362 0.2023 0.2089 0.2289 0.2307 0.2257 0.2460 0.1858 0.1932 0.2397 0.1961 To evaluate the positional accuracy, the shoreline of the represented lakes produced from different land cover mapping techniques was compared with the boundary of the corresponding lakes in the ground data image. Several points along the shoreline of a represented lake were selected. These points were vectorised and compared with the closest points along the shoreline of the corresponding lake in the ground data. As an example, Fig. 4 shows boundary fitting for different land cover mapping techniques on Lake 25 from the image in Fig. 3. The boundary of the represented lakes was overlaid with the boundary of the corresponding lake in the ground data. Fig. 4 shows that object represented by traditional HC have an unrealistic jagged and pixelated. Using only a single MODIS image, the boundaries represented all the HNN and PS techniques appeared to be irregular and serrated. However, the Table 1 provides an average of the difference for area (in km2) compared to the ground data. Overall, the proposed HNN2 applied on the fused MODIS time series images produced the lowest error for the area measurement with the average error was 0.1290 km2, while the PS(5) applied on a single MODIS image produced the highest error with 0.2009 km2. By using the fused image, all the techniques demonstrated decreasing in the error for the area measurement of the lakes. Table 2 provides an average of the difference for perimeter (in meter) compared to the ground data. For the perimeter measurement, the PS(5) applied on the fused image produce the lowest error with 0.6732 m, while the HNN2 59 2017 IEEE 8th Control and System Graduate Research Colloquium (ICSGRC 2017), 4 - 5 August 2017, Shah Alam, Malaysia application of the time series MODIS images produced different results than by using a single MODIS image. For the time-series MODIS images, the lake boundaries in the HNN(E) tended to be smoother and less irregular when compared to the lake boundaries in the HNN(A). This trend is not apparent in the single MODIS image case in which both the HNN(E) and the HNN(A) produced irregular lake boundaries. The advantage of the HNN(E) in retaining the smoothness of lake boundaries and the advantage of the HNN(A) in representing small lakes were exploited by combining the these two techniques in the proposed HNN2. lakes represented appeared to be blocky and jagged. The positional accuracy for the represented lakes in the fused image was higher than that in the single image. The boundary of the represented lakes in the fused image appears to be smoother and less irregular than the represented lakes in the single image. An example of this scenario is shown in Figure 4. VI. CONCLUSION This paper presents the application of different SRM techniques on MODIS 250 m imagery. To characterize the shape of the lakes, standard measures such as area, perimeter and compactness were used. In general, the HNN2 technique applied on a time series MODIS images was more accurate than the other techniques in predicting the area of the represented lakes, while the PS(5) was more accurate in predicting the perimeter and the compactness of the represented lakes. On the positional accuracy, the HNN(E) was more accurate than the other techniques as the boundary of the lakes represented was smooth, which resemblance the boundary of the lakes in the ground data. As a conclusion, all the SRM techniques applied on a time series MODIS images produced better accuracy than when applied on a single MODIS image. The vector difference between the points along the boundary of the represented lakes and the points along the boundary of the corresponding lakes in the ground data was measured to produce positional accuracy information, as shown in an example in Fig. 5. The root mean squared error (RMSE) of the position error of the selected 30 lakes (Fig. 3) was calculated and the result is presented in Table 4. In general, the HNN(E) applied on the fused image produced the highest positional accuracy compared with other techniques with the average RMSE produced was 82.98 m. The proposed HNN 2 technique that was applied on the fused image produced the second highest positional accuracy with the average RMSE was 85.00 m. The HNN(A) applied on a single image produced the lowest positional accuracy with the average of the RMSE was 145.93 m. ACKNOWLEDGMENT Funding by the Ministry of Higher Education Malaysia under the research grant FRGS/2/2013/TK03/UKM/02/5 is gratefully acknowledged. REFERENCES  D.S. Boyd and G.M. Foody, “An overview of recent remote sensing and GIS based research in ecological informatics,” Ecological Informatics, vol. 6(1), pp. 25-36, January 2011.  E.A. Addink, D. Jong, S.A. Davis, V. Dubyanskiy, L.A. Burdelov, and H. Leirs, “The use of high-resolution remote sensing for plague surveillance in Kazakhstan,” Remote Sensing of Environment, vol. 114(3), pp. 674-681, March 2010.  W. Buermann, S. Saatchi, T.B. Smith, B.R. Zutta, J.A. Chaves, B. Mila, and C.H. Graham, “Predicting species distributions across the Amazonian and Andean regions using remote sensing data,” Journal of Biogeography, vol. 35(7), pp. 1160-1176, July 2008.  G.M. Foody, “Status of land cover classification accuracy assessment,” Remote Sensing of Environment, vol. 80(1), pp. 185-201, April 2002.  T.A. Warner, M.D. Nellis, and G.M. Foody, (2009), “Remote sensing scale and data selection issues,” in The SAGE Handbook of Remote Sensing. T.A. Warner, M.D. Nellis, and G.M. Foody, Eds. London, Sage, 2009, pp. 3-17.  C. Woodcock and A. Strahler, “The factor of scale in remote sensing,” Remote Sensing of Environment, vol. 21(3), pp. 311-332, April 1987.  B. Tso, and P.M. Mather, Classification Methods for Remotely Sensed Data, London: Taylor and Francis, 2009.  A.J. Tatem, H.G. Lewis, P.M. Atkinson, and M.S. Nixon, “Superresolution target identification from remotely sensed image using a Hopfield neural network,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39(4), pp. 781-796, April 2001.  F. Ling, Y. Du, F. Xiao, H. Xue, and S. Wu, “Super-resolution landcover mapping using multiple sub-pixel shifted remotely sensed images,” International Journal of Remote Sensing, vol. 31(19), 50235040, September 2010.  Y.F. Su, G.M. Foody, A.M. Muad, and K.S. Cheng, “Combining Hopfield neural network and contouring methods to enhance superresolution mapping,” IEEE Journal of Selected Topics in Applied Earth Fig. 5. Positional error along the boundary of lake 25. Table 4. Average difference of RMSE on boundary fitting (m). Technique Single image Fused image HC PS (1) PS (5) HNN (E) HNN (A) HNN2 117.44 109.83 138.39 141.80 145.93 141.48 88.44 85.38 82.98 145.25 85.00 A general trend was notable from the Table 4 that the PS(5), HNN(E), and HNN2 techniques, all were applied on the fused image, tended to produce highly accurate positional prediction as the RMSE values from the three techniques were relatively low than that of the other techniques. The boundary of the lakes represented by the PS(5), HNN(E) and the HNN 2 were generally smoother and less irregular than the boundary of the lakes represented by other techniques, notably the PS(1) and the HNN(A). The average RMSE of the boundary fitting of the hard classifier was also high because the boundary of the 60 2017 IEEE 8th Control and System Graduate Research Colloquium (ICSGRC 2017), 4 - 5 August 2017, Shah Alam, Malaysia       Observations and Remote Sensing, vol. 5(5), pp. 1403-1417 October 2012. X. Li, Y. Du, F. Ling, Q. Feng, and B. Fu, “Superresolution mapping of remotely sensed image based on Hopfield neural network with anisotropic spatial dependence model,” IEEE Geoscience and Remote Sensing Letters, vol. 11(7), pp. 1265–1269, July 2014. P.M. Atkinson, “Sub-pixel target mapping from soft-classified, remotely sensed imagery,” Photogrammetric Engineering and Remote Sensing. 71(7), 839-846, July 2005. Y.F. Su, G.M. Foody, A.M. Muad, and K.S. Cheng, “Combining pixel swapping and contouring methods to enhance super-resolution mapping,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5(5), pp. 1428-1437 October 2012. D. Nigussie, R. Zurita-Milla, and J.G.P.W, Clevers, “Possibilities and limitations of artificial neural networks for subpixel mapping of land cover,” International Journal of Remote Sensing, vol. 32(22), pp. 72037226, November 2011. J.P. Ardila, V.A. Tolpekin, W. Bijker, and A. Stein, “Markov-randomfield-based super-resolution mapping for identification of urban trees in VHR images,” Journal of Photogrammetry and Remote Sensing, vol. 66(6), pp. 762-775, November 2011. X. Xu, Y. Zhong, L. Zhang, and S. Member, “Adaptive subpixel mapping based on a multiagent system for remote-sensing imagery,”       61 IEEE Transactions on Geoscience and Remote Sensing, vol. 52(2), pp. 787–804, February 2014. Q. Wang, P.M. Atkinson, and W. Shi, “Indicator cokriging-based subpixel mapping without prior spatial structure information,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53(1), pp. 309323, January 2015. J. Hu, Y. Ge, Y. Chen, and D. Li, “Super-resolution land cover mapping based on multiscale regularization,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8(5), pp. 20312039, May 2015. J.C. Bezdek, R. Ehrlich, and W. Full, “FCM: The fuzzy c-means clustering algorithm,” Computer and Geosciences, vol 10(2-3), pp. 191203, December 1984 A.M. Muad and G.M. Foody, “Impact of land cover patch size on the accuracy of patch area representation in HNN-based super-resolution mapping,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5(5), pp. 1418-1427, October 2012. M.G. Sicairos, S.T. Thurman, and J.R. Fienup, “Efficient subpixel image registration algorithms,” Optics Letters, vol. 33(2), pp. 156-158, January 2008. B.S. Reddy and B.N. Chatterji, “An FFT-based technique for translation, rotation, and scale invariant image registration,” IEEE Transactions on Image Processing, vol. 5(8), pp. 1266-1271, August 1996.