Salesforce BigObjects

What is a BigObject?

Two years ago I started working with Big Objects, a Salesforce pilot that looked very interesting for me. And now that it seems that General Available time is closer, I would like to write about them.

Please, take into account that a Salesforce pilot could be discarded like Data Pipeline that it would not be GA at the end.

Big Objects is a new type of objects that Salesforce provides. Maybe the most important thing I can tell is that they do not count against the storage limit. Well, you could say that External Objects already do.

You can learn more about External Objects here.

Yes, you are right. But this time, data lives in your organisation instead of in an external storage and this means that you can work with them like with Custom / Standard objects, but if you go to Salesforce Storage screen, you would not find them.

Big Objects let you store and manage large amounts of data.

Use Case

But before starting with the explanation, let’s talk about the Use Case.

As a company that creates software, we have stories in order to develop new functionality. Before delivering them, we need to pass a code review, so that, we ensure the best to our end users.

Captura de pantalla 2017-07-07 a las 12.21.47

But how much storage should I use to keep code review records in our Production org? Should I remove them? If so, what about auditing?

BigObjects is your solution, so that, you can move old records to a new Code Review History BigObject record. Let’s focus for now on this new History object.

How to Create a BigObject?

Unfortunately you cannot do it in a declarative way. You need to define them via Metadata API and then do a deployment via cmd or workbench.

Before doing it, take into account:

  1. Indexes are required and in order to include them, you need to be on API 39.0 onwards.
  2. Once you deploy one, you cannot make any modification like amend a field type, change label, etc. However in a declarative way you can do small changes like the label and API name.
<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
  <deploymentStatus>Deployed</deploymentStatus>
  <fields>
    <fullName>CanBeMerged__c</fullName>
      <label>Can Be Merged</label>
      <length>80</length>
      <type>Text</type>
    </fields> 
  <fields>
  <fullName>CodeReviewDate__c</fullName>
    <label>Code Review Date</label>
    <type>DateTime</type>
    <required>true</required>
  </fields>
  <fields>
    <fullName>Comments__c</fullName>
    <label>Comments</label>
    <length>255</length>
    <type>Text</type>
  </fields>
  <fields>
    <fullName>Employee__c</fullName>
    <label>Employee</label>
    <referenceTo>Employee__c</referenceTo>
    <relationshipName>Employees</relationshipName>
    <required>true</required>
    <type>Lookup</type>
  </fields>
  <fields>
    <fullName>Score__c</fullName>
    <label>Score</label>
    <precision>1</precision>
    <scale>0</scale>
    <type>Number</type>
  </fields>
  <fields>
    <fullName>Story__c</fullName>
    <label>Story__c</label>
    <referenceTo>Story__c</referenceTo>
    <relationshipName>Stories</relationshipName>
    <required>true</required>
    <type>Lookup</type>
  </fields>
  <indexes>
    <type>PRIMARY</type> 
    <fullName>Test1CodeReviewPK</fullName>
    <fields>
      <name>Story__c</name>
      <sortDirection>DESC</sortDirection> 
    </fields>
    <fields>
      <name>Employee__c</name>
      <sortDirection>DESC</sortDirection>
    </fields>
    <fields>
      <name>CodeReviewDate__c</name>
      <sortDirection>DESC</sortDirection>
    </fields>
  </indexes>
  <label>Test1 Code Review History</label>
  <pluralLabel>Test1 Code Reviews History</pluralLabel>
</CustomObject>

Once you deploy it into your organization, you would find something like this under BigObject entry:

I decided to name is as Test1 Code Review History because once it is created I cannot modify it, and I didn’t want to use Code Review History label for now.

Captura de pantalla 2017-07-14 a las 13.03.54.png

What can we highlight?

  1. The API name ends with __b instead of __c or __x like custom and external objects.
  2. We will not find Standard fields.
  3. We can create as many custom fields as we want but right now it only supports these field types: DateTime, Lookup, Number, Text and Long TextArea. What about the rest? Try to be creative. For instance a Checkbox can be converted into number or text.
  4. You will not find any other section like PageLayout or Buttons for now.
  5. You will not find Triggers section. It is not allowed for now.
  6. You will not be allowed to create a custom tab related to this object. That is related to previous point and the PageLayout. To visualise Big Object records, you need to create a visualforce page or a lightning component.
  7. You can determine its CRUD and FLS via Profile or Permission Sets during the deploy process or in a declarative way. However for now BigObjects can be only created and read.

Captura de pantalla 2017-07-14 a las 13.44.36

How Can I Create Records?

There are different ways to create BigObject record, like using a csv file, use APIs like Bulk API or even Async SOQL, another Pilot I will talk about in a future entry.

In any case, here, I will focus on Apex.

As I mentioned above, you can only Read or Create records, and nowadays, you cannot use simple DML operations, there is a new one: database.insertImmediate(record)

Test1CodeReview__b cr = new Test1CodeReview__b();
cr.CanBeMerged__c = 'False';
cr.CodeReviewDate__c = System.today();
cr.Comments__c = 'I found a SOQL inside of a loop.';
cr.Employee__c = 'a6j24000000fxSL';
cr.Score__c = 0; 
cr.Story__c = 'a6k24000000k9bN';
database.insertImmediate(cr);

But something pretty cool that it was not available at the beginning, it is the fact we can use this method, to also update records. Actually it works like upsert. If I make a call where the record has a primary key that doesn’t exist, it would create a new record as we can see in the below image.

Captura de pantalla 2017-07-14 a las 14.28.04

However if the value already is in the platform, it will make a modification.

Test1CodeReview__b cr = new Test1CodeReview__b();
cr.CanBeMerged__c = 'False';
cr.CodeReviewDate__c = System.today();
cr.Comments__c = 'I found a SOQL inside of a loop.';
cr.Employee__c = 'a6j24000000fxSL';
cr.Score__c = 1; //Change to 1
cr.Story__c = 'a6k24000000k9bN';
database.insertImmediate(cr);

Captura de pantalla 2017-07-14 a las 14.35.49

If I come back to the original use case, I would like to move CodeReview__c records to Test1CodeReviewHistory__b records. In order to do that I only need to retrieve the record I need and create a new BigObject entry.

CodeReview__c cr = [Select Id, CanBeMerged__c, 
                           CodeReviewDate__c, Comments__c, 
                           Employee__c, Score__c, Story__c 
                    From CodeReview__c 
                    Limit 1];

String canBeMerged = cr.CanBeMerged__c == true ? 'True' : 'False';

Test1CodeReview__b crh = new Test1CodeReview__b();
crh.CanBeMerged__c = canBeMerged;
crh.CodeReviewDate__c = cr.CodeReviewDate__c;
crh.Comments__c = cr.Comments__c;
crh.Employee__c = cr.Employee__c;
crh.Score__c = cr.Score__c; 
crh.Story__c = cr.Story__c;
database.insertImmediate(crh);

How Can I Visualise Records?

As we have already mentioned, we cannot create a custom tab related to a BigObject and show all records. However we can query BigObjects, so what about if we create a custom page for that?

First of all we would need a controller. Really simple one. We have a method to retrieve all records and a get and set to show in a list what we retrieve.

public with sharing class CodeReviewController
{
   private static List<Test1CodeReview__b> codeReviewHistoryList; 
   
   public CodeReviewController()
   {
      setCodeReviewHistoryList(calculateCodeReviewHistoryList());
   } 

   public List<Test1CodeReview__b> calculateCodeReviewHistoryList()
   {
      return [SELECT Id, CanBeMerged__c, CodeReviewDate__c, 
                     Comments__c, Employee__c, Score__c, 
                     Story__c                
               FROM Test1CodeReview__b];
   }

   public static List<Test1CodeReview__b> getCodeReviewHistoryList()
   {
      return codeReviewHistoryList;
   }

   public static void setCodeReviewHistoryList(List<Test1CodeReview__b> value)
   {
      codeReviewHistoryList = value;
   }
}

Secondly, we have the visualforce page. Something to highlight is that right now, the standardController attribute doesn’t work, so we can only create a page with a customController. 

<apex:page showHeader="true" sidebar="true" controller="CodeReviewController">
   <apex:form>
      <apex:pageBlock title="Code Review History Records">
         <apex:pageBlockSection title="Code Review History" columns="1" collapsible="false">
	    <apex:pageBlockTable value="{!codeReviewHistoryList}" var="crHistory">
               <apex:column headerValue="Ready To Merge?">
	          <apex:outputField value="{!crHistory.CanBeMerged__c}"/>
               </apex:column>
	       <apex:column headerValue="Score">
	          <apex:outputField value="{!crHistory.Score__c}"/>
	       </apex:column>
               <apex:column headerValue="Comments">
	          <apex:outputField value="{!crHistory.Comments__c}"/>
               </apex:column>
               <apex:column headerValue="Story">
	          <apex:outputField value="{!crHistory.Story__c}"/>
               </apex:column>
               <apex:column headerValue="Reviewer">
                  <apex:outputField value="{!crHistory.Employee__c}"/>
               </apex:column>
               <apex:column headerValue="Code Review Date">
                  <apex:outputField value="{!crHistory.CodeReviewDate__c}"/>
               </apex:column>
            </apex:pageBlockTable>
         </apex:pageBlockSection>
      </apex:pageBlock>
   </apex:form>
</apex:page>

And the result is this one that includes the record created from scratch and the one that is retrieved from CodeReview__c object.

Captura de pantalla 2017-07-17 a las 9.14.52

Summary

I would like to share this table that summarise BigObjects and compare with Custom Objects. But before, I would like to also highlight that this new object type can be included in a package. You would find under CustomObject section:

Captura de pantalla 2017-07-17 a las 9.16.06

Summary Table:

Feature Custom Object Big Object
Creation Manual
Metadata
Metadata
API name myObject__c myObject__b
Track Activities
Track Field History, etc.
Options Available Options No Available
Field Types All Text ; Date/Time ; Lookup
Number ; Long Text Area
Able to edit fields Yes Yes (with restrictions)
Able to delete fields Yes No
Triggers; Field Sets; etc Options Available Options No Available
Reports Yes No
How to Populate records All CSV file
API (Bulk, SOAP)
Apex
Async SOQL
Can I amend a record? Yes Yes (with restrictions)
Can I see data creating a Tab Yes No
For free? Yes No — Talk with Salesforce
Storage It count against storage It doesn’t count against storage

21 thoughts on “Salesforce BigObjects

  1. Thank you Augustina for the great article.
    How do we enable big objects in dev Edition org. Do we need to create a case with salesforce. where do we need to do this

    Like

  2. That upsert doesn’t make sense to me. How can the platform know that you are updating the information instead of inserting a new one, if you are not providing an Id? To me, a upsert call will always insert a new record if you don’t pass an Id.

    Like

    • Sorry for the late response. Not sure if you realized about indexed fields. They work as primary key. So when you try to insert a record, it looks the PK and if it finds a record with same values as PK, it update the record.

      Like

  3. Muchas gracias Agustina, me ha parecido una entrada muy completa y muy interesante. Es una característica que en mi opinión, en cuanto deje de tener las limitaciones actuales, puede ser un bombazo.
    Nuevamente gracias,
    Esteve

    Like

  4. Perdona Agustina, respecto a las limitaciones que comentabas en tu artículo, añadiría que ciertos operadores en las SOQL no están permitidos, te paso la frase exacta de la documentación: The !=, LIKE, NOT IN, EXCLUDES, and INCLUDES operators are not valid in any query.

    Like

    • Gracias Esteve. Es muy gratificante tener feedback y si además añade información para otros es genial.
      Después de Dreamforce ’17 publicaré otro sobre Async SOQL. Espero que te guste también.

      Like

  5. Pingback: Big Objects: Contenedores de Datos – un quiero y quizás puedo de almacenamiento masivo en Salesforce – Artículos sobre Salesforce

  6. Good Introduction to Big Data..It’s really helpful.
    Could you please tell me how can we use the big object to Track Field History for custom/standard sobject.
    I have an idea that we can create a big object and relate to a custom or standard object and then create an after update trigger on that object that will insert the field changes into the custom big object…but not sure that is a good solution.

    Liked by 1 person

    • Thanks!! Happy it was useful for you. Actually there is a Standard Big Object called
      FieldHistoryArchive is a standard big object that stores data as part of the Field Audit Trail
      product. Not sure if this can help. Otherwise, yes it make sense your idea about the after update trigger.

      Like

      • You can find an example on How can I create records? section. That piece of code create records on a simple method. However I’m not so sure that you can use it inside of a custom object trigger. I have asked product manager and waiting for his response.

        Like

  7. Good article on big object.
    Could you please let me know whether we can insert any records to bigobject from any trigger(standard/Custom ).

    As i have a requirement to store changes history(any field value changes) of opportunity split to a big object. where we are planning to use after insert/update/delete trigger on opportunity split to save all the information on abigobject.

    So i am not sure whether is it possible or not. Please help.

    Like

    • Thanks Sudipta. Actually you can create records via Database.InsertInmediate. You can find more information on How Can I Create Records? section.

      To be honest, I have not tried in a trigger. I guess there is no issue. At the end it is apex code, but keep in mind bulkification

      Like

  8. Pingback: 10 thoughts about Big Objects – My Salesforce adventure

  9. Error in inserting records in Big objects via Apex

    I have used Database.insertImmediate(objOppSplitHistories); method to insert/update records into OppSplitHistories (my custom big object).
    Here I have used the above method in apex class and called that method from oppsplit trigger to pass the old map/new map info for saving opp splits changes. But that is throwing the below error. I have observed if I run the same piece of code in ‘Execute Anonymous Window’ then data is being inserted in to custom big object but despite using insertImmediate method it is not working through trigger.

    Error: Update failed. First exception on row 0 with id 049c0000002JaopAAC; first error: UNKNOWN_EXCEPTION, EXTERNAL_OBJECT_UNSUPPORTED_EXCEPTION: Apex triggers can’t reference the Big Object BigObjectCounter: []

    It seems we cannot populate any bigobject through trigger . The considerations only says that we cannot write Triggers on Big object but it does not say that we should not call the InsertImmediate method from a Trigger (on a Custom object). Its not clear. Any help or clarity will be much appreciated.

    Like

    • You can find an example on How can I create records? section. That piece of code create records on a simple method. However I’m not so sure that you can use it inside of a custom object trigger. I have asked product manager and waiting for his response.

      Like

  10. I have talked with PM and he confirmed that it is not posible to create a big object record inside a trigger. However a workaround could be to put the insertImmediate() call into a queueable method – in this way it separates the operation from the ongoing trigger’s transaction.

    Like

  11. Pingback: Big Object field types workarounds – My Salesforce adventure

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s