Corporate tech blogs are often about putting out showcase worthy pieces of work that gets created within the four walls of the organisation. And yet, it is common knowledge that organizations have hundreds if not thousands of services, enterprise source code repositories that holds tens of millions of lines of code, most of which doesn’t come anywhere close to being showcase worthy. While this can be hidden from public view, it is a burden that the organisation carries and weighs down up the technical leadership team.
Any person who has spent even a few days as a full-time software developer knows that it is very easy to start a new project and have it written in an idealistic manner. The same set of people would also speak of the futility of trying to achieve similar quality over the tens of millions of lines of code that exists in the internal repository.
Press enter or click to view image in full size
The rest of the blog is a real-world story on how Gen-AI gives hope of solving this problem that has eluded us forever.
The test subject: 36 lines of code
We, at Glance had an incident in production a few ago that was detected and mitigated in the way that we always have been doing. As I went through the post-mortem report, I noticed that just a couple of lines of code written that had a naive error had caused the incident. The source file in question is part of one of oldest, biggest, and most actively changed git repository which means that this is naturally susceptible to code decay. Note that code decay is very different from tech debt, and I am not talking about tech debt here.
My attention was not on how the changes that caused the incident slipped through. Instead, I wanted to see if the source file that contained this error was now free of any other defects. To start with, I had difficulty in reading the entire file as it seemed to have coding style that I was not used it. I ended up choosing one of the methods from that file to explore the state of AI and the findings shook me up quite a lot. Here is the piece of code on which all the experiments were performed. This code segment is now fondly known as “36 lines of code” in a lot of internal discussions
private void populateEmojiMapping(Map<String, List<GlancePair>> glancePairMap){
try {
for (String glanceId: glancePairMap.keySet()) {
List<GlancePair> glancePairs = glancePairMap.get(glanceId);
Set<String> filteredGlanceIds = glancePairs.stream().filter(x -> !x.getMeta().isRemoved()
&& x.getMeta().getEuIdentifier() == GlanceEUIdentifier.ANDROID_TV)
.map(x -> x.getData().getGlanceId())
.collect(Collectors.toSet());
if (!filteredGlanceIds.isEmpty()) {
log.info("fetching interactions getting glance-emoji mapping for glanceIds: {}",
String.join(",", filteredGlanceIds));
val client = this.httpUtils.getHttpClient();
URIBuilder uriBuilder = new URIBuilder(getGlanceEmojiMappingUrl);
uriBuilder.addParameter("glanceIds", String.join(",", filteredGlanceIds));
HttpGet request = new HttpGet(uriBuilder.build());
val httpResponse = client.execute(request);
String entityString = EntityUtils.toString(httpResponse.getEntity());
log.info("response from interaction-service: {}", entityString);
Set<GlanceEmojiMapping> glanceEmojiMappings = objectMapper.readValue(entityString,
objectMapper.getTypeFactory().constructCollectionType(Set.class, GlanceEmojiMapping.class));
for (GlanceEmojiMapping glanceEmojiMapping : glanceEmojiMappings) {
glanceEmojiMap.put(glanceEmojiMapping.getGlanceId(), glanceEmojiMapping);
}
// delete removed glances
Set<String> removedGlances = glancePairs.stream().filter(x -> x.getMeta().isRemoved())
.map(x -> x.getData().getGlanceId())
.collect(Collectors.toSet());
glanceEmojiMap.keySet().removeAll(removedGlances);
} else {
log.info("Emoji: No data needs to be fetched!");
}
}
} catch (Exception e) {
log.error("Exception occurred for fetching & setting emojis", e);
}
}The human test
My claim on the state of AI when it comes to coding is this: on most tasks, the AI will outperform a human in terms of quality, let alone speed. A good way to test this claim is for you to read this piece of code yourself and find all possible issues. The time limit for reviewing this code is 5 minutes. If you think this is too short, here is the sobering fact: an actual human review would not spend more time than 5 minutes on a seemingly simple 36 lines of code if this came in a pull request review. Apologies to non-Java programmers as you cannot participate in this test.
Enter the AI tutor
The first prompt
My first question was not to fix anything but simply pose this question
Find potential non-obvious problems with this codeNote that I am choosing to be totally ignorant the programming language, the popular libraries used and even the specifics of the problem at hand.
Here is the response from my tool
Let me analyze this code for potential non-obvious issues:Resource Management:
* The HTTP client and response resources aren't properly closed using try-with-resources
* EntityUtils.toString() consumes the entire response into memory at once, which could be problematic for large responses
* The HTTP connection isn't explicitly released back to the connection pool
Error Handling:
* The catch block catches all exceptions and only logs them, silently swallowing errors
* There's no distinction between different types of failures (network, parsing, validation)
* The method continues with potentially incomplete/invalid data after errors
Concurrency:
* glanceEmojiMap appears to be a shared resource but has no synchronization
* The keySet().removeAll() operation isn't atomic with the puts, creating a potential race condition
Performance:
* Creates multiple streams for the same data source (glancePairs)
* Makes HTTP requests in a loop without batching
* Potentially inefficient repeated string joining operations
Input Validation:
* No validation of URL parameters or maximum number of glanceIds
* No checks for null values in the map or lists
* No validation of response data before processing
Impressed? It doesn’t stop here. It gives out the improvised code with the following explanation
Key improvements in the refactored version:Resource Management:
* Uses try-with-resources for proper cleanup
* Explicitly handles HTTP resources
* Sets character encoding for response parsing
Error Handling:
* Separates different types of exceptions
* Handles HTTP status codes
* Validates input data
Concurrency:
* Synchronizes access to shared map
* Makes updates atomic
* Prevents race conditions
Performance:
* Processes glances in batches
* Single pass for data processing
* Reuses collections where possible
Robustness:
* Validates input data
* Handles null values
* Limits batch sizes
* Sets proper HTTP headers
Maintainability:
* Better logging with appropriate levels
* Clearer code structure
* More descriptive variable names
private void populateEmojiMapping(Map<String, List<GlancePair>> glancePairMap) {
if (glancePairMap == null || glancePairMap.isEmpty()) {
log.warn("Empty glance pair map provided");
return;
} // Process all glances in one batch instead of per-key loop
Set<GlancePair> allGlancePairs = glancePairMap.values().stream()
.flatMap(List::stream)
.collect(Collectors.toSet());
// Single pass to separate active and removed glances
Map<Boolean, Set<String>> partitionedGlances = allGlancePairs.stream()
.collect(Collectors.groupingBy(
pair -> pair.getMeta().isRemoved(),
Collectors.mapping(
pair -> pair.getData().getGlanceId(),
Collectors.toSet()
)
));
Set<String> activeGlanceIds = partitionedGlances.getOrDefault(false, Collections.emptySet())
.stream()
.filter(id -> id != null && !id.isEmpty())
.collect(Collectors.toSet());
if (activeGlanceIds.isEmpty()) {
log.info("No active glances to process");
return;
}
// Calculate batch sizes to avoid URL length limits
int maxGlanceIdsPerRequest = 50; // Adjust based on URL length limits
List<Set<String>> glanceIdBatches = new ArrayList<>();
Iterator<String> iterator = activeGlanceIds.iterator();
while (iterator.hasNext()) {
Set<String> batch = new HashSet<>();
while (iterator.hasNext() && batch.size() < maxGlanceIdsPerRequest) {
batch.add(iterator.next());
}
glanceIdBatches.add(batch);
}
// Process batches
for (Set<String> batch : glanceIdBatches) {
try (CloseableHttpClient client = this.httpUtils.getHttpClient()) {
URIBuilder uriBuilder = new URIBuilder(getGlanceEmojiMappingUrl);
uriBuilder.addParameter("glanceIds", String.join(",", batch));
HttpGet request = new HttpGet(uriBuilder.build());
request.setHeader(HttpHeaders.ACCEPT, ContentType.APPLICATION_JSON.getMimeType());
try (CloseableHttpResponse response = client.execute(request)) {
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != HttpStatus.SC_OK) {
log.error("Failed to fetch emoji mappings. Status: {}", statusCode);
continue;
}
String entityString = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("Response for batch size {}: {}", batch.size(), entityString);
Set<GlanceEmojiMapping> mappings = objectMapper.readValue(entityString,
objectMapper.getTypeFactory().constructCollectionType(Set.class, GlanceEmojiMapping.class));
synchronized (glanceEmojiMap) {
// Update mappings atomically
mappings.forEach(mapping -> {
if (mapping != null && mapping.getGlanceId() != null) {
glanceEmojiMap.put(mapping.getGlanceId(), mapping);
}
});
// Remove inactive glances atomically
Set<String> removedGlances = partitionedGlances.getOrDefault(true, Collections.emptySet());
glanceEmojiMap.keySet().removeAll(removedGlances);
}
}
} catch (IOException e) {
log.error("Network error while fetching emoji mappings for batch", e);
} catch (URISyntaxException e) {
log.error("Invalid URI for emoji mapping request", e);
} catch (JsonProcessingException e) {
log.error("Failed to parse emoji mapping response", e);
} catch (Exception e) {
log.error("Unexpected error in emoji mapping processing", e);
}
}
}
This shows that not only did I get fixes for the code, but I also got an excellence commit message for this changeset as well.
Rot prevention
Get Arvind Jayaprakash’s stories in your inbox
Join Medium for free to get updates from this writer.
As mentioned earlier, my goal is prevention of code-rot and not just a one time reversal. The first line of defence against regressions is having the right kind of tests in place. My next prompt however was not to ask for unit tests.
Is this code unit testable?Note that this is often misunderstood as prompt engineering i.e. expertise in prompting. In reality, the prompt I chose comes from proficiency in programming that allowed me to see that this code does not easily lend itself to testing. I am calling this out not to brag about my skills but to instead let you, the reader, know that there is still a role for human expertise.
Here is the fixed-up code. I am omitting the explanation offered.
// Interface for the service functionality
public interface EmojiMappingService {
Map<String, GlanceEmojiMapping> fetchAndProcessEmojiMappings(Map<String, List<GlancePair>> glancePairMap);
}// Data fetching interface
public interface EmojiMappingClient {
Set<GlanceEmojiMapping> fetchEmojiMappings(Set<String> glanceIds) throws EmojiMappingException;
}
// Configuration class
@Value
public class EmojiMappingConfig {
int maxBatchSize;
int maxRetries;
}
// Custom exception
public class EmojiMappingException extends Exception {
public EmojiMappingException(String message) {
super(message);
}
public EmojiMappingException(String message, Throwable cause) {
super(message, cause);
}
}
// Main implementation
@Service
@Slf4j
public class DefaultEmojiMappingService implements EmojiMappingService {
private final EmojiMappingClient emojiClient;
private final EmojiMappingConfig config;
@Inject
public DefaultEmojiMappingService(
EmojiMappingClient emojiClient,
EmojiMappingConfig config) {
this.emojiClient = Objects.requireNonNull(emojiClient);
this.config = Objects.requireNonNull(config);
}
@Override
public Map<String, GlanceEmojiMapping> fetchAndProcessEmojiMappings(
Map<String, List<GlancePair>> glancePairMap) {
Objects.requireNonNull(glancePairMap, "glancePairMap cannot be null");
// Process active and removed glances
GlancePartitionResult partitionResult = partitionGlances(glancePairMap);
if (partitionResult.getActiveGlanceIds().isEmpty()) {
log.info("No active glances to process");
return Collections.emptyMap();
}
// Fetch and process in batches
Map<String, GlanceEmojiMapping> resultMap = new HashMap<>();
List<Set<String>> batches = createBatches(partitionResult.getActiveGlanceIds());
for (Set<String> batch : batches) {
try {
Set<GlanceEmojiMapping> mappings = emojiClient.fetchEmojiMappings(batch);
updateMappings(resultMap, mappings);
} catch (EmojiMappingException e) {
log.error("Failed to process batch: {}", batch, e);
}
}
// Remove mappings for removed glances
resultMap.keySet().removeAll(partitionResult.getRemovedGlanceIds());
return resultMap;
}
@VisibleForTesting
GlancePartitionResult partitionGlances(Map<String, List<GlancePair>> glancePairMap) {
Set<String> activeGlanceIds = new HashSet<>();
Set<String> removedGlanceIds = new HashSet<>();
glancePairMap.values().stream()
.flatMap(List::stream)
.filter(Objects::nonNull)
.forEach(pair -> {
String glanceId = pair.getData().getGlanceId();
if (glanceId != null) {
if (pair.getMeta().isRemoved()) {
removedGlanceIds.add(glanceId);
} else if (isValidGlance(pair)) {
activeGlanceIds.add(glanceId);
}
}
});
return new GlancePartitionResult(activeGlanceIds, removedGlanceIds);
}
@VisibleForTesting
List<Set<String>> createBatches(Set<String> glanceIds) {
List<Set<String>> batches = new ArrayList<>();
Iterator<String> iterator = glanceIds.iterator();
while (iterator.hasNext()) {
Set<String> batch = new HashSet<>();
while (iterator.hasNext() && batch.size() < config.getMaxBatchSize()) {
batch.add(iterator.next());
}
batches.add(batch);
}
return batches;
}
private boolean isValidGlance(GlancePair pair) {
return pair.getMeta() != null &&
!pair.getMeta().isRemoved() &&
pair.getMeta().getEuIdentifier() == GlanceEUIdentifier.ANDROID_TV;
}
private void updateMappings(Map<String, GlanceEmojiMapping> resultMap,
Set<GlanceEmojiMapping> newMappings) {
newMappings.stream()
.filter(mapping -> mapping != null && mapping.getGlanceId() != null)
.forEach(mapping -> resultMap.put(mapping.getGlanceId(), mapping));
}
}
// HTTP implementation of the client
@Service
public class HttpEmojiMappingClient implements EmojiMappingClient {
private final HttpUtils httpUtils;
private final ObjectMapper objectMapper;
private final String mappingUrl;
@Inject
public HttpEmojiMappingClient(
HttpUtils httpUtils,
ObjectMapper objectMapper,
@Named("emojiMappingUrl") String mappingUrl) {
this.httpUtils = Objects.requireNonNull(httpUtils);
this.objectMapper = Objects.requireNonNull(objectMapper);
this.mappingUrl = Objects.requireNonNull(mappingUrl);
}
@Override
public Set<GlanceEmojiMapping> fetchEmojiMappings(Set<String> glanceIds)
throws EmojiMappingException {
try (CloseableHttpClient client = httpUtils.getHttpClient()) {
URIBuilder uriBuilder = new URIBuilder(mappingUrl);
uriBuilder.addParameter("glanceIds", String.join(",", glanceIds));
HttpGet request = new HttpGet(uriBuilder.build());
request.setHeader(HttpHeaders.ACCEPT, ContentType.APPLICATION_JSON.getMimeType());
try (CloseableHttpResponse response = client.execute(request)) {
return processResponse(response);
}
} catch (Exception e) {
throw new EmojiMappingException("Failed to fetch emoji mappings", e);
}
}
private Set<GlanceEmojiMapping> processResponse(CloseableHttpResponse response)
throws IOException, EmojiMappingException {
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != HttpStatus.SC_OK) {
throw new EmojiMappingException("Invalid response status: " + statusCode);
}
String entityString = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
return objectMapper.readValue(entityString,
objectMapper.getTypeFactory().constructCollectionType(Set.class, GlanceEmojiMapping.class));
}
}
@Value
class GlancePartitionResult {
Set<String> activeGlanceIds;
Set<String> removedGlanceIds;
}
@ExtendWith(MockitoExtension.class)
class DefaultEmojiMappingServiceTest {
// ... previous test methods ... private Map<String, List<GlancePair>> createTestGlancePairs() {
Map<String, List<GlancePair>> testData = new HashMap<>();
// Create one list of glance pairs for category "featured"
List<GlancePair> featuredPairs = Arrays.asList(
createGlancePair("glance-1", false, GlanceEUIdentifier.ANDROID_TV),
createGlancePair("glance-2", false, GlanceEUIdentifier.ANDROID_TV),
// Add an invalid pair that should be filtered out
createGlancePair("glance-3", false, GlanceEUIdentifier.IOS)
);
testData.put("featured", featuredPairs);
// Create another list for category "trending"
List<GlancePair> trendingPairs = Arrays.asList(
createGlancePair("glance-4", false, GlanceEUIdentifier.ANDROID_TV),
// Add a removed pair that should be filtered out
createGlancePair("glance-5", true, GlanceEUIdentifier.ANDROID_TV),
// Add a null check case
null
);
testData.put("trending", trendingPairs);
return testData;
}
private Map<String, List<GlancePair>> createMixedGlancePairs() {
Map<String, List<GlancePair>> testData = new HashMap<>();
List<GlancePair> mixedPairs = Arrays.asList(
// Active glances
createGlancePair("active-glance-1", false, GlanceEUIdentifier.ANDROID_TV),
createGlancePair("active-glance-2", false, GlanceEUIdentifier.ANDROID_TV),
// Removed glances
createGlancePair("removed-glance-1", true, GlanceEUIdentifier.ANDROID_TV),
createGlancePair("removed-glance-2", true, GlanceEUIdentifier.ANDROID_TV),
// Different platform glance
createGlancePair("ios-glance-1", false, GlanceEUIdentifier.IOS),
// Edge cases
createGlancePair(null, false, GlanceEUIdentifier.ANDROID_TV),
createGlanceWithNullMeta("glance-null-meta")
);
testData.put("mixed", mixedPairs);
return testData;
}
private Set<GlanceEmojiMapping> createTestEmojiMappings() {
return new HashSet<>(Arrays.asList(
createEmojiMapping("glance-1", "😊", "happy"),
createEmojiMapping("glance-2", "🎮", "gaming"),
createEmojiMapping("active-glance-1", "🌟", "star"),
createEmojiMapping("active-glance-2", "🎯", "target")
));
}
private GlancePair createGlancePair(String glanceId, boolean isRemoved, GlanceEUIdentifier euIdentifier) {
GlancePair pair = new GlancePair();
// Create and set GlanceData
GlanceData data = new GlanceData();
data.setGlanceId(glanceId);
pair.setData(data);
// Create and set GlanceMeta
GlanceMeta meta = new GlanceMeta();
meta.setRemoved(isRemoved);
meta.setEuIdentifier(euIdentifier);
pair.setMeta(meta);
return pair;
}
private GlancePair createGlanceWithNullMeta(String glanceId) {
GlancePair pair = new GlancePair();
GlanceData data = new GlanceData();
data.setGlanceId(glanceId);
pair.setData(data);
// Deliberately leave meta as null
pair.setMeta(null);
return pair;
}
private GlanceEmojiMapping createEmojiMapping(String glanceId, String emoji, String description) {
GlanceEmojiMapping mapping = new GlanceEmojiMapping();
mapping.setGlanceId(glanceId);
mapping.setEmoji(emoji);
mapping.setDescription(description);
return mapping;
}
// Additional test methods using this test data
@Test
void shouldHandleNullValues() {
// Given
Map<String, List<GlancePair>> input = createTestGlancePairs();
// When
GlancePartitionResult result = service.partitionGlances(input);
// Then
assertThat(result.getActiveGlanceIds())
.contains("glance-1", "glance-2", "glance-4")
.doesNotContain("glance-3", "glance-5", null);
}
@Test
void shouldProperlyPartitionMixedGlances() {
// Given
Map<String, List<GlancePair>> input = createMixedGlancePairs();
// When
GlancePartitionResult result = service.partitionGlances(input);
// Then
assertThat(result.getActiveGlanceIds())
.contains("active-glance-1", "active-glance-2")
.doesNotContain("removed-glance-1", "removed-glance-2", "ios-glance-1", null, "glance-null-meta");
assertThat(result.getRemovedGlanceIds())
.contains("removed-glance-1", "removed-glance-2")
.doesNotContain("active-glance-1", "active-glance-2", "ios-glance-1", null, "glance-null-meta");
}
@Test
void shouldHandleEmptyAndNullValues() throws EmojiMappingException {
// Given
Map<String, List<GlancePair>> input = createMixedGlancePairs();
Set<GlanceEmojiMapping> mockResponse = createTestEmojiMappings();
when(emojiClient.fetchEmojiMappings(any())).thenReturn(mockResponse);
// When
Map<String, GlanceEmojiMapping> result = service.fetchAndProcessEmojiMappings(input);
// Then
assertThat(result)
.containsKeys("active-glance-1", "active-glance-2")
.doesNotContainKeys("removed-glance-1", "removed-glance-2", "ios-glance-1", null, "glance-null-meta");
}
}
I have not provided the prompt that was used for generating the test data. This is left as an exercise for the reader.
Which LLM achieved all of this?
This is actually a moot question or will become one by say, February of 2025. The best LLM for coding has been changing multiple times in the past few months. All of the above code was generated using Claude Sonnet via the standard browser interface. But what was showcased here is going to be the median level of performance of any LLM within a matter of few weeks.
What does this mean
Tutor v/s assistant
Note that AI is no longer the intern a.k.a. assistant helping us do mundane tasks. It is actually a veteran that knows how to look for well-known pitfalls that one learns about through years of experience. This inversion of relationship between the human and the AI is an inflection point that is not known well enough and this is what changes everything.
Here are a few examples where the AI has shown expertise
- Introduction of CloseableHttpResponse
- Knowing the need and correct usage of @VisibleForTesting
- Knowing that this is a Spring code and hence using @Service, @Value, @Inject
- Generating test data and exercising it via a test framework
- …
But AI cannot …
- It is natural for a veteran developer to point out that they can outperform AI on some highly complex task
- It is natural for a veteran developer to point out that this is low complexity code and hence one should not extrapolate the abilities of AI
But the human didn’t …
It is also a fact that despite the above two things being true, the real-world production code that could have been fixed by AI wasn’t fixed. And that a human would never fix it as it seems like too much work to do against too large a codebase and applying these kinds of fixes without AI would mean that there would be no time left to build new features.
And that is the opportunity that we all need to understand. We no longer need to put up with our ugly ducklings. Our randomly sampled code from a population of tens of millions of lines of code can look as beautiful as the carefully selected code that we put out on our corporate blogs at one fiftieth of what it used to cost us in the past.