Saturday, July 15, 2017

elasticsearch 2.2 does not allow dot in fieldname

Today we will take another look at the elasticsearch 2.2 exception. Below is the stacktrace,

 [2016-06-26 00:06:48,265][DEBUG][action.admin.indices.mapping.put] [node1] failed to put mappings on indices [[my_index]], type [my_type]  
 MapperParsingException[Field name [foo.bar] cannot contain '.']  
     at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:276)  
     at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:221)  
     at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:138)  
     at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:119)  
     at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:100)  
     at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:435)  
     at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:257)  
     at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)  
     at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:458)  
     at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:762)  
     at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)  
     at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)  
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  
     at java.lang.Thread.run(Thread.java:745)  

It turn out that the map key cannot contain dot, read the method parsePropeties below, I supposed this break a lot and yet it can be easily replace with an underscore before index.

     protected static void parseProperties(ObjectMapper.Builder objBuilder, Map<String, Object> propsNode, ParserContext parserContext) {  
       Iterator<Map.Entry<String, Object>> iterator = propsNode.entrySet().iterator();  
       while (iterator.hasNext()) {  
         Map.Entry<String, Object> entry = iterator.next();  
         String fieldName = entry.getKey();  
         if (fieldName.contains(".")) {  
           throw new MapperParsingException("Field name [" + fieldName + "] cannot contain '.'");  
         }  

These are some url to help you better understand why dot is removed in elasticsearch 2.x

  • https://discuss.elastic.co/t/field-name-cannot-contain/33251/48
  • https://github.com/elastic/elasticsearch/commit/aed1f68e494c65ad50b98a3e0a7a2b6a794b2965        
  • https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking_20_mapping_changes.html#_field_names_may_not_contain_dots
  • https://github.com/elastic/elasticsearch/issues/15951
  • https://github.com/elastic/elasticsearch/issues/14594
  • https://github.com/elastic/elasticsearch/issues/14957

but yet, in elasticsearch 5, dot is allow back? oh my gawd, what's your take?

No comments:

Post a Comment