What is a BigObject?

Two years ago I started working with Big Objects, a Salesforce pilot that looked very interesting for me. And now that it seems that General Available time is closer, I would like to write about them.

Please, take into account that a Salesforce pilot could be discarded like Data Pipeline that it would not be GA at the end.

Big Objects is a new type of objects that Salesforce provides. Maybe the most important thing I can tell is that they do not count against the storage limit. Well, you could say that External Objects already do.

You can learn more about External Objects here.

Yes, you are right. But this time, data lives in your organisation instead of in an external storage and this means that you can work with them like with Custom / Standard objects, but if you go to Salesforce Storage screen, you would not find them.

Big Objects let you store and manage large amounts of data.

Use Case

But before starting with the explanation, let’s talk about the Use Case.

As a company that creates software, we have stories in order to develop new functionality. Before delivering them, we need to pass a code review, so that, we ensure the best to our end users.

But how much storage should I use to keep code review records in our Production org? Should I remove them? If so, what about auditing?

BigObjects is your solution, so that, you can move old records to a new Code Review History BigObject record. Let’s focus for now on this new History object.

How to Create a BigObject?

Unfortunately you cannot do it in a declarative way. You need to define them via Metadata API and then do a deployment via cmd or workbench.

Before doing it, take into account:

1. Indexes are required and in order to include them, you need to be on API 39.0 onwards.
2. Once you deploy one, you cannot make any modification like amend a field type, change label, etc. However in a declarative way you can do small changes like the label and API name.

<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
  <deploymentStatus>Deployed</deploymentStatus>
  <fields>
    <fullName>CanBeMerged__c</fullName>
      <label>Can Be Merged</label>
      <length>80</length>
      <type>Text</type>
    </fields> 
  <fields>
  <fullName>CodeReviewDate__c</fullName>
    <label>Code Review Date</label>
    <type>DateTime</type>
    <required>true</required>
  </fields>
  <fields>
    <fullName>Comments__c</fullName>
    <label>Comments</label>
    <length>255</length>
    <type>Text</type>
  </fields>
  <fields>
    <fullName>Employee__c</fullName>
    <label>Employee</label>
    <referenceTo>Employee__c</referenceTo>
    <relationshipName>Employees</relationshipName>
    <required>true</required>
    <type>Lookup</type>
  </fields>
  <fields>
    <fullName>Score__c</fullName>
    <label>Score</label>
    <precision>1</precision>
    <scale>0</scale>
    <type>Number</type>
  </fields>
  <fields>
    <fullName>Story__c</fullName>
    <label>Story__c</label>
    <referenceTo>Story__c</referenceTo>
    <relationshipName>Stories</relationshipName>
    <required>true</required>
    <type>Lookup</type>
  </fields>
  <indexes>
    <type>PRIMARY</type> 
    <fullName>Test1CodeReviewPK</fullName>
    <fields>
      <name>Story__c</name>
      <sortDirection>DESC</sortDirection> 
    </fields>
    <fields>
      <name>Employee__c</name>
      <sortDirection>DESC</sortDirection>
    </fields>
    <fields>
      <name>CodeReviewDate__c</name>
      <sortDirection>DESC</sortDirection>
    </fields>
  </indexes>
  <label>Test1 Code Review History</label>
  <pluralLabel>Test1 Code Reviews History</pluralLabel>
</CustomObject>

Once you deploy it into your organization, you would find something like this under BigObject entry:

I decided to name is as Test1 Code Review History because once it is created I cannot modify it, and I didn’t want to use Code Review History label for now.

What can we highlight?

1. The API name ends with __b instead of __c or __x like custom and external objects.
2. We will not find Standard fields.
3. We can create as many custom fields as we want but right now it only supports these field types: DateTime, Lookup, Number, Text and Long TextArea. What about the rest? Try to be creative. For instance a Checkbox can be converted into number or text.
4. You will not find any other section like PageLayout or Buttons for now.
5. You will not find Triggers section. It is not allowed for now.
6. You will not be allowed to create a custom tab related to this object. That is related to previous point and the PageLayout. To visualise Big Object records, you need to create a visualforce page or a lightning component.
7. You can determine its CRUD and FLS via Profile or Permission Sets during the deploy process or in a declarative way. However for now BigObjects can be only created and read.

How Can I Create Records?

There are different ways to create BigObject record, like using a csv file, use APIs like Bulk API or even Async SOQL, another Pilot I will talk about in a future entry.

In any case, here, I will focus on Apex.

As I mentioned above, you can only Read or Create records, and nowadays, you cannot use simple DML operations, there is a new one: database.insertImmediate(record)

Test1CodeReview__b cr = new Test1CodeReview__b();
cr.CanBeMerged__c = 'False';
cr.CodeReviewDate__c = System.today();
cr.Comments__c = 'I found a SOQL inside of a loop.';
cr.Employee__c = 'a6j24000000fxSL';
cr.Score__c = 0; 
cr.Story__c = 'a6k24000000k9bN';
database.insertImmediate(cr);

But something pretty cool that it was not available at the beginning, it is the fact we can use this method, to also update records. Actually it works like upsert. If I make a call where the record has a primary key that doesn’t exist, it would create a new record as we can see in the below image.

However if the value already is in the platform, it will make a modification.

Test1CodeReview__b cr = new Test1CodeReview__b();
cr.CanBeMerged__c = 'False';
cr.CodeReviewDate__c = System.today();
cr.Comments__c = 'I found a SOQL inside of a loop.';
cr.Employee__c = 'a6j24000000fxSL';
cr.Score__c = 1; //Change to 1
cr.Story__c = 'a6k24000000k9bN';
database.insertImmediate(cr);

If I come back to the original use case, I would like to move CodeReview__c records to Test1CodeReviewHistory__b records. In order to do that I only need to retrieve the record I need and create a new BigObject entry.

CodeReview__c cr = [Select Id, CanBeMerged__c, 
                           CodeReviewDate__c, Comments__c, 
                           Employee__c, Score__c, Story__c 
                    From CodeReview__c 
                    Limit 1];

String canBeMerged = cr.CanBeMerged__c == true ? 'True' : 'False';

Test1CodeReview__b crh = new Test1CodeReview__b();
crh.CanBeMerged__c = canBeMerged;
crh.CodeReviewDate__c = cr.CodeReviewDate__c;
crh.Comments__c = cr.Comments__c;
crh.Employee__c = cr.Employee__c;
crh.Score__c = cr.Score__c; 
crh.Story__c = cr.Story__c;
database.insertImmediate(crh);

How Can I Visualise Records?

As we have already mentioned, we cannot create a custom tab related to a BigObject and show all records. However we can query BigObjects, so what about if we create a custom page for that?

First of all we would need a controller. Really simple one. We have a method to retrieve all records and a get and set to show in a list what we retrieve.

public with sharing class CodeReviewController
{
   private static List<Test1CodeReview__b> codeReviewHistoryList; 
   
   public CodeReviewController()
   {
      setCodeReviewHistoryList(calculateCodeReviewHistoryList());
   } 

   public List<Test1CodeReview__b> calculateCodeReviewHistoryList()
   {
      return [SELECT Id, CanBeMerged__c, CodeReviewDate__c, 
                     Comments__c, Employee__c, Score__c, 
                     Story__c                
               FROM Test1CodeReview__b];
   }

   public static List<Test1CodeReview__b> getCodeReviewHistoryList()
   {
      return codeReviewHistoryList;
   }

   public static void setCodeReviewHistoryList(List<Test1CodeReview__b> value)
   {
      codeReviewHistoryList = value;
   }
}

Secondly, we have the visualforce page. Something to highlight is that right now, the standardController attribute doesn’t work, so we can only create a page with a customController. 

<apex:page showHeader="true" sidebar="true" controller="CodeReviewController">
   <apex:form>
      <apex:pageBlock title="Code Review History Records">
         <apex:pageBlockSection title="Code Review History" columns="1" collapsible="false">
	    <apex:pageBlockTable value="{!codeReviewHistoryList}" var="crHistory">
               <apex:column headerValue="Ready To Merge?">
	          <apex:outputField value="{!crHistory.CanBeMerged__c}"/>
               </apex:column>
	       <apex:column headerValue="Score">
	          <apex:outputField value="{!crHistory.Score__c}"/>
	       </apex:column>
               <apex:column headerValue="Comments">
	          <apex:outputField value="{!crHistory.Comments__c}"/>
               </apex:column>
               <apex:column headerValue="Story">
	          <apex:outputField value="{!crHistory.Story__c}"/>
               </apex:column>
               <apex:column headerValue="Reviewer">
                  <apex:outputField value="{!crHistory.Employee__c}"/>
               </apex:column>
               <apex:column headerValue="Code Review Date">
                  <apex:outputField value="{!crHistory.CodeReviewDate__c}"/>
               </apex:column>
            </apex:pageBlockTable>
         </apex:pageBlockSection>
      </apex:pageBlock>
   </apex:form>
</apex:page>

And the result is this one that includes the record created from scratch and the one that is retrieved from CodeReview__c object.

Summary

I would like to share this table that summarise BigObjects and compare with Custom Objects. But before, I would like to also highlight that this new object type can be included in a package. You would find under CustomObject section:

Summary Table: