Low-level robust image features have been proven to be efficient representation for a variety of computer vision task such as visual recognition. But as the visual recognition tasks such as scene classification and object detection become more challenging and complicated, the semantic gap between low-level feature and the semantic concept descriptor of the scene images increases. Even though the existing convolutional networks produce robust features, they fail to capture the complex semantic contents of the image. Scene images are comprising of differ- ent concepts located spatially with varying size. Semantic multinomial (SMN) is a semantic representation of an image corresponds a posterior probabilities vector of concepts. In this work, we proposed Concept neural network (CoNN) to generate SMN representation. Concept neural network enables the SMN representation to hold the complex semantic contents of an image. It is necessary to have ground truth (true) concept labels to train the CoNN. In this paper we address the issue of unavailability to concept labels. We proposed to use pseudo-concepts in the absence of true concept labels. CNN features are used to discover the pseudo-concepts and train the Concept neural network. We studied the effectiveness of the proposed SMN representations for scene classification task using standard datasets.