Hello there! My name is Annie (Suantak Niangneihoi), recently welcomed as part of the SAP Analytics Product Management team. It is an honor to be writing my first blog as someone who empowers and supports the developer community.

I’ve been working on converting visualization extensions from the previous SAP Lumira SDK versions to the newer and recommended SAP Web IDE versions. This blog post is supplementary to the original Football Heatmap viz extension blog written by Mustafa Aydogdu in conjunction to the wonderful extension he developed. In this blog, I will mention step-by-step how I migrated the existing extension from SAP Lumira SDK to SAP Web IDE – which is technically rebuilding the extension but doesn’t really involve too much effort since it is a simple process. You can find out more about migrating extensions in general in Dong Pan’s blog post.

The reason why we’re so interested in migrating our old extensions to SAP Web IDE is mainly for compatibility with the latest version of SAP Lumira. The more we enhance a product, the more we also need to enhance the add-ons associated with it – extensions. This is why our experts strongly recommend using SAP Web IDE to create new extensions.

In my blog, I aim to provide a concise explanation of the migration process, making it easy to follow even for someone new to SAP Web IDE, using the football heatmap use case as an example.

Let us proceed to the fun part now!

STEP 1: Create the project

1. Create a new project in SAP Web IDE (Fileà New à Project from template).
2. Select the Visualization Extension template.
3. Type in the project name (FootballHeatmap)
4. Create the profile for the extension:
   1. Give the extension a name (Football Heatmap)
   2. Extension ID: sap.viz.ext.footballheatmap
   3. Company name: Write the name of the developer, not SAP
   4. Add a URL and description if desired.
5. The original extension was developed using DIV, not SVG, so select Using DIV. You may remove the title and legend since they are not explicitly required.
6. Load the sample dataset (recordsView.csv), choose the X and Y coordinates as dimensions and Value as the measure.
7. Add the above attributes to the measure and dimension sets as required.
8. Click on Finish. Your project will be created.

STEP 2: Load the necessary external files

Here, we use only one external JS file, i.e. heatmap2.js. Import it to SAP Web IDE into the footballheatmap subfolder (FootballHeatmap/bundles/sap/viz/ext/footballheatmap). This file should be in parallel with the footballheatmap-bundle.js file.

(Right click on the footballheatmap folder--> Import --> From File System)

The file structure should look like this:

STEP 3: Visualize while you work

Click on the VizPacker Quick Preview icon on the top of the right hand menu panel in SAP Web IDE.

This will allow you to preview your extension at every step of development, depending on the changes you make as you progress.

STEP 4: Edit the render script (render.js)

Load the resource at the top level define() function (the first line of code in the render.js file)

Note: It is very important that you include the heatmap2.js file within the [ ] and just writing it as ‘heatmap2’ instead of ‘heatmap2.js’

    Next, copy the D3 rendering code from the old render.js into the render function

var render = function(data, container) {

                  //paste code here

};

return render;

});

The code starts from the container.selectAll(‘svg’).remove(); line

Now before we proceed, we add a few more lines of code to render the canvas appropriately as Dong Pan mentions in his blog post.

Add the following lines above the original code as highlighted in the screenshot below:

Scroll down a bit on the code. You will find the require function that calls the intializeHeatmap() function.

In the old render.js file, replace the highlighted section below:

with this highlighted section:

What we are doing here is simply replacing the way the external resource, i.e. heatmap2.js, is called. We replace the directory structure (we don’t need the path since we have already imported the resource in the appropriate directory earlier), with just the name of the resource file (without the .js file extension).

Note: File path structure is extremely important. A project may not render due to a simple error in file path even though code may be a 100% correct. This is a speed bump for many developers

    Now save the render.js file

STEP 5: Edit the CSS file (default.css)

Just copy all of the CSS code from the old version of the extension to the default.css file in the SAP Web IDE project.

Save the file

STEP 6: See it render in SAP Web IDE

By now you should be able to see a preview of the extension rendering in the VizPacker Quick Preview section of SAP Web IDE

You can also Run it and see what it will look like for different screen sizes.

STEP 7: Pack the extension

The extension looks good! Now let’s pack it so that we can use it in SAP Lumira!

Click on Pack. This will download the extension as an installable .zip file ready for use.

STEP 8: Install and use in SAP Lumira

1. Install the extension as you would normally in SAP Lumira using the Extension Manager.
2. Import the recordsView.csv dataset.
3. Arrange the measures and dimensions as required.
4. Check out the extension!

Now, doesn’t that look lovely?

Thank you all for taking the time to read my blog post. As I continue working on migration of visualization extensions from SAP Lumira SDK to SAP Web IDE, there will be many more blog posts to come, on how to migrate each of the open-source extensions our developers at SAP have come up with.

Please feel free to add your comments, suggestions or ideas.

On this page, we summarize the best practices for publishing your Open Source extensions for Lumira. These best practices aim at making your effort and contribution as successful as possible:

• High quality code: to make the extension usable
• Hight quality blog: to get the right attention
• High quality presence on github: to make the extension easy to consume.

If there are any best practices you would like to add, please feel free to do so in the comments or send me an email: suantak.niangneihoi@sap.com

SAP Lumira: Flexibility through Custom Visualization Extensions

SAP Lumira is a great data visualization tool that comes with various bar charts, pie charts, cross tables, geo-maps, to name just a few. In most situations, these standard charts are very useful and powerful visuals to represent your data and turn data into information. But, what if you need something very special to your business, for example showing customer traffic in a retail store, player activity in a sports game or website click analytics?

Charts for such unique and special visuals are not offered out of the box with SAP Lumira: Instead, we have a very powerful and flexible option: Custom Visualization Extensions.

In order to show how an unusual visualization can be created for Lumira, I have decided to create football heatmaps, which shows a football player’s activity on a football field. You can see these kind of visuals on Fifa.com or football analysis websites. The heatmap visualization is not limited to sports: It can easily be transferred to uses cases in other industries, for example Retail.

The Data

Data which is shown in this kind of visualizations is usually collected by HD cameras, beacons or RFIDs, which are able to track an object’s  movements. In our case, it’s the activity of a football player, but the concept is easily expandable to customers’ activity in a retail store: anywhere, where GPS sensors cannot be attached to the object. The cameras generate tracking data with object id and position (football player, customer, …). The position of the tracked objects are usually represented as X and Y coordinates.
Please don’t get confused with geo-spatial coordinates like latitude and longitude, this position system is different from geographic coordinates. Actually the idea is the same: representing a position in a 2-dimensional world with 2 parameters, but it is very specific to field, industry or camera producer. An example of dataset is available on GitHub.

Let’s Start with the Layout

I have started creating my extension from the football pitch. It is basically a combination of a green rectangle, circles and lines. I have chosen to draw the pitch with D3 code instead of using an image. In this way when you change the size of the visual on stories within SAP Lumira (‘Compose tab’), the pitch will be resized perfectly without any pixelation or distortion.

Drawing lines, circles or green field with d3 is really easy. However, when it comes to semi-circles(arcs) it was more difficult. Main difficulty is when you resize the pitch, edges of penalty arc exceeds the lines. In order to create a perfect penalty arc, you need to calculate the center angle of the arc and dynamically change this angle according to the size of the pitch.  That calculation took me back in time, to my college years: I refreshed my trigonometric knowledge and as a result I created a perfect pitch, which preserves its shape even if it is resized.

If you would like to use this extension for a retail store or for a different scenario (like the one below - SAP Forum İstanbul visitor heatmap), you can have an image of your store, instead of d3 coded football pitch. I have an html div called “pitchContainer” inside the code, which contains d3 created layout. With a little HTML effort, you can locate your image inside this div.

Visitor Heatmap Created by Infonomi (blog by Daniel Zimmer)

 

 

Locating the Heatmap on Top of the Layout

After creating the pitch, it is time to create the heatmap and locate it to the right place on the pitch. In order to create the heatmap, I used an opensource javascript library called heatmap.js. With this library it is really easy to create dynamic heatmaps for lots of use cases.

Heatmap.js requires 2 important objects; an html div element to create the heatmap inside it, and the data that will be visualized.

 

 

As I said before it is crucial to find the right place for locating the heatmap. Data formats will vary for different sports, industries, even for different football data providers. Locating this different

formatted data into right place has one trick: D3 scale functions. If you know the maximum and minimum data points (location x,y) of your data, you can assign these values to yMin, yMax, xMin and xMax variables, otherwise extension will calculate min and max of dataset dynamically. The scale below gets the real data point (location x or y) and converts it to the right point on the football pitch according to current size of the field (fieldHeight and fieldWidth).

If you want to use a different layout, probably you will use the same code for creating the heatmap. The only difference will be the image or drawing that you will be using instead of the football pitch. If you use the same systematic for width and height, most of the work will be done automatically.

Final visual is a combination of the pitch and the heatmap. It is possible to filter according to team, player or time. You can analyze a football game interactively and create a storyboard to show multiple players on the same canvas.

 

 

 

 

Extension In Action

Let’s start playing. As I said before you need players and location of these players during the game.

1. Download sample data set from github, which contains, team, player and location information.
2. Load dataset into Lumira.
3. Select Football Heatmap extension.
4. For this extension measure is not very important, but you have to add one. In this case add Count.
5. For the location dimesion add X and Y dimensions as shown.
6. You can filter by team or players to see how they perform.

 

Or

 

You can simply download sample lumira file (Football Heatmap Demo.lums) and open it with lumira.

 

 

Just a Note For the Big Amount of Data Points

Since these kind of visuals require a huge number of points you may need to increase the default maximum number of points that can be visualized. You can find “Dhilo.maxvizdatasetsize” parameter under SAPLumira.ini file and increase that number. I increased it to 100k for my case.

 

Power of Extensions

Your business may be different from others and it may require special analysis. Lumira provides you a rich portfolio of features and visuals in order to make “Trusted Data Discovery”. When you need special type of visuals for your work, you are not limited to, out of the box ones, you can create your own visuals as you imagine.

 

It is easy, powerful and flexible. Enjoy the power of extensions.

P.S. Special thanks to Suantak Niangneihoi for migrating extension to WEB IDE.

The next installment of the DataGenius Challenge (formerly DataGeek) is now underway. I thought it would be fun to take a look back at some of the more memorable entries. Here are my favorite DataGenius entries over the years:

The Future of David Hasselhoff– One of the more creative entries, take a look at where the future career of “The Hoff” with these predictive insights.

Visualizing Gangnam Style– A jaw-dropping custom visualization of the popular Gangnam style video rendered in Lumira.

Who is the Real Jon Snow– Game of Thrones fans will find this entry an insightful attempt to uncover the real life Jon Snow with geo-spatial analysis.

Sharknado Social Media Analysis Using SAP Predictive Analysis– Tap into the social-verse with this DataGenius entry visualizing unstructured data from the cult movie Sharknado. Sharks, tornadoes, and predictive analytics, what else do you want?

MS Dhoni, analyzing the win machine– Take a look at the career of this cricket legend and uncover the insights which make MS Dhoni the Michael Jordan of the cricket world.

In addition to having fun with data, this years DataGenius challenge is about supporting and uncovering insights for real-world causes. Check out how the DataGenius Challenge and Autism Speaks are teaming up to support insights for Autism research. What is your favorite entry? Check out other data driven stories and join this year’s DataGenius Challenge.

Stay in touch with the latest data driven stories on Twitter, and follow me @nicfish for more DataGenius news

During the past month, i've been testing the connectivity of SAP Lumira with Spark on Azure was able to create great analytics.

here are the steps you need to follow :

1. Download and install Simba ODBC with SQL connector

2 .Configure the Simba ODBC with SQL connector

Guide: Spark ODBC Driver Quickstart

Configuration used in our testing:

3. Download and install SAP Lumira Spark extension

4. Launch SAP Lumira

5. Acquire Data:

6. Datasource selection:

Hint: it might be hidden behind the “add new dataset screen”

6. Press OK

7. Enter the password

• Optional : Test the connection

8. Press Ok, then ok

9. You can either select an entire table as the data set

10. Or fitter the data with SQL statements

Hint: if you face issue here, you may wanna try the query within beeline to make sure the syntax is ok and nothing is wrong with the cluster

11. Re-enter the password.

Hint: it might be hidden behind the “add new dataset screen”

enjoy

After seeing this Nic Smith's blog Data Genius Challenge for Autism Insight, I tried accessing the data sources listed but without success; one site said I could only access it if I was a researcher (which I am not).  If others have success accessing data, please share your data sources.  I am not an intuitive user so I could be missing something.

I happened to find this National Center for Biotechnology Information and searched for publications on autism.  It returned over 30K publications since 1976.  I thought I would use some text analysis with SAP Lumira to learn about what is being discussed about autism.

The text analysis brought back over 80K rows looking at text by people, person, title, product and various organization types.

I used Lumira Desktop 1.27.1.  I always start with suggested visualizations in Lumira, as I am fairly lazy with the tools and it's a nice place to start.

Counts by type was a suggested visualization, with most of the text analysis resulting in types of people (72K) and the rest organizations (5K).

Counts of mentions by governmental organization are shown above.  It is not surprising to see the CDC mentioned but the top mentioned is Dobyns of Seattle Children's Hospital.

It's not really a surprise either that the AMA (American Medical Association) is the most mentioned in the articles under the medical science organization type.

The above shows the most mentioned in terms of products.  Murine is really a model of autism spectrum disorders.

Here's a separate visualization on Lumira Cloud.  Of course it looks best on the iPad using SAP BusinessObjects Mobile

On the iPad you can annotate.  I didn't look too closely at the publications for top counts by people - it seemed to mostly be by race; text analysis likely things "Magyar" means Hungarian but searching there has been autism research performed by people named "Magyar".

So for fun I tried the cross tab on the iPad and linking back to Nic's Data Genius post:

I am looking forward to what others share about autism in future posts.

Reference

Add these SAP Lumira Sessions at #SABOUC ASUG Conference in Austin

Announcing ASUG SAP TechEd Pre-Conference Sessions - Open for Registration

LUMIRA LIVE

Monthly SAP Lumira

Community Newsletter

• What's New in SAP Lumira 1.25:
  A number of new features have been introduced in the latest release of SAP Lumira to improve usability. Read the summary and review features in this tutorial.
• David McCandless Visualization in SAP Lumira
  Best-selling author, data journalist, creator of InformationIsBeautiful.net and DataGenius David McCandless released a never-before seen Internet of Things visualization at this year’s SAPPHIRE NOW conference. Download a version of it and try it yourself in SAP Lumira!
• Create a Predictive Calculation
  SAP Lumira visualizations can now leverage some of the functionality from the SAP Predictive Analytics software to forecast values for future time periods. In this example, we will add predictive calculations to a line chart using both SAP Predictive Analytics forecasting and triple exponential smoothing.
  
  

Pay Increase Summary

Who's smiling the most on pay day? Visualize your compensation analysis with SAP Lumira.

 

Events

• June 2 – 18: SAP Analytics Innovation Days
• June 16–18: SAP Insider BI 2015
  In-depth education, including comprehensive workshops, lectures, demos, and panel discussions presented by top experts at SAP, leading customers, and the most authoritative consultants in SAP's BI and BW arenas.

Webinars

• Recording: SAP Lumira, server for BI Platform
• May 27: Create Relevant Visualizations and Beautiful Infographics with SAP Lumira
• June 3: SAP Lumira and Big Data

Connect with SAP Lumira:

Read Previous Issues:

April 2015
March 2015
November 2014
September 2014

Hello, All

Animal husbandry plays very vital role in India’s economy. Being the country where 72% of the people employed in the agriculture field which includes the animal husbandry , it is very important for India to understand several aspects which contributes the growth of agriculture and animal husbandry. There are several aspects and benefits in doing the animal husbandry but one prominent among them is the milk production. Now we shall analyze the milk production in the year of 2010 and 2011 in India using SAP Lumira.

Before getting into the analysis please note that all numbers are in ‘000 tonne and source has been taken from Indian government website.

Let’s start with looking at the total production. In the year of 2009-2010 the total production was 112,540 (000) tonne and it was increased by 9304 (000) tonnes (8.2 percent).  But it is Rajasthan was almost became double in the milk production and 20% increase in the Tamilnadu state. All other states were almost same as the previous year.

The highest milk production state in 2011 was UP followed by Rajasthan, AP and Gujarat. The overall composition from different states has well explained in the attached chart.  Another important point to note down is both in 2010 and 2011, buffalos produce more milk than cows.

Here are different visualization which gives more explanation and open more analysis.

(I am sorry that I could not upload the complete story here. I tried uploading the visualizations however I have options to upload only three files! If somebody can educate me on this, please do so. Thank you in advance.)

Thanks,

Prasanna.

I LOVE how easy it is to download, install, and share data driven stories with SAP Lumira. I LOVE it so much I decided to make a video about it. Our marketing team needs to quickly share analysis and insights for our marketing initiatives and campaigns. SAP Lumira can help prepare, visualize, and share data driven stories within our organization and our key stakeholders.

This short demo shows how easy it is to download and install Lumira Server so that you can extend the power of agile visualization with your teams. In just 10 mins you can be up and running on your own personal laptop without the need for any expensive hardware or additional technical support. Now that's how to Run Simple.

Download SAP Lumira at www.sap.com/lumira and see what interesting insights you can come up with. Share and see what others have done with SAP Lumira and other SAP Technology by checking out the DataGenius Challenge www.sap.com/datagenius.

Follow me on Twitter for more insights on analytics and data discovery and share your experience with telling data driven stories in your organization.

Many of the SAP Lumira extensions for data visualization that have been developed and open-sourced, were developed in the SAP Lumira SDK, using the Lumira VizPacker Utility. However, ever since the Vizpacker plugin for SAP Web IDE has reached functional parity, SAP has been recommending using the SAP Web IDE environment for development of visualization extensions for SAP Lumira. Raz Korn explains in his blog the perks of using SAP Web IDE over SAP Lumira VizPacker Utility, and why it is highly recommended for someone who wants to build cool, attractive data visualization extensions for SAP Lumira.

I have been hugely involved in the process of migrating our existing viz extensions that have been created in the SAP Lumira SDK, to SAP Web IDE. I have explained an example of how I migrated Mustafa Aydogdu’s Football Heatmap extension in my blog post as well.

Here, I would like to highlight some challenges I faced during the process of migration from Lumira SDK to Web IDE. The general steps to be followed in the migration process are pretty simple and straightforward as explained in Dong Pan’s blog post. However, there are some special cases where the extension requires external resources like an external JavaScript file, or an image file that may needed to be loaded to render certain parts of the chart. This is what I would like to focus on in this article. I believe and hope that this will help many developers out there who are trying to build their Lumira visualization extensions in Web IDE or even trying to migrate old extensions previously created with the old Lumira VizPacker Utility-based SDK.

In my blog post on migrating the Football Heatmap extension, I did mention that an external file by the name of heatmap2.js was required as part of the package. Similarly, as an example in this blog, I will use the example of a project called dummyproject with an external file called external.js

Let us now see in three simple steps how to import external or third party anonymous modules or resources to be used as part of the new extension to be build in Web IDE.

Step 1: Put the file parallel to the *-bundle.js file

For example, in the screenshot snippet above, we see that our project bundle JS file is called dummyproject-bundle.js. Generally, we have <projectname>-bundle.js, contained in the <projectname> folder. All the external resources should be imported in parallel to this project bundle file.
To do this, right click on the containing folder (dummyproject folder in this case) à Import à From File System. This way, all the desired resources are loaded in the same containing folder as the project bundle file, in parallel to it.

Step 2: Load the resource at the top-level define()

Now this step is very important because even though we have loaded the resources into the project file structure, we still need to call those resources via our code.
First and foremost, call the dependencies at the define() function at the top of your render.js file.

Notice how we call external instead of external.js

Step 3: Call the resource via a require function

In the main source code, we add a require function as shown below:

After following the steps mentioned above, we can now proceed to building our Lumira extension in Web IDE as usual!

Special thanks to: Raz Korn, George Wang& Andrew Zhu for their assistance and support in putting together all these useful tips.

Note: These instructions are ONLY to provide information about how to include external resources and dependencies into a Web IDE project, and NOT about how to create a Web IDE-based Lumira visualization extension in general.

Hi everyone.

Galigeo for Lumira is coming back in a fresh new version, which introduces a new feature that might be of interest to many of you.

Whether you are already using Galigeo for Lumira or not, you can find the latest version on the SAP Analytics Market Place.

You’ll also be able to find anything useful about it, such as how to edit the embedded catalog to display your custom layers, here and there.

What is this new feature about?

Haha, you've got it. The title of this blog says it all .

We have decided to call it Smart Drill Down (or more generally speaking, Smart Drill), firstly because it will allow anyone to drill down or up on its layers on the map, and secondly because it does not need any configuration from the user in order to take effect. Not even defining a hierarchy of any kind in Lumira. It will works with any custom or default data.

To demonstrate how it works, let's look at a real world example.

We’ll put ourselves in the shoes of a drinks manufacturing company.

This company has an abudance of data about the pubs and restaurants it delivers its products, such as how many products are delivered each week, net income, turnover, … like any company that uses Lumira I suppose.

Well,please note that the following example derives from a real case study from one of our clients but the data values have been modified for the sake of this example.

Well, let’s get started!

I want to display a potential turnover over Paris(France) city. I have several layers for it.

• The first one is made of polygons that display sectors. It’s technical sectors, mostly sales ones
• The second one is also made of polygons that display subsectors, also called subareas. Each sector from the first layer contains several subsectors
• The third layer is made of points, and represents the pubs/restaurants where my company is delivering fresh products.

With previous version of Galigeo For Lumira, if I’d chosen a single indicator (Potential Turnover in K€) and then drop the my three layers into the Geolayers input, this is what I would have got so far.

Well, it already provides good insights for sure but there are several drawbacks :

• The layers are all displayed at the same time by default
• I have no way to emulate drill down or drill up onto the layers

So I would have to manually filter the data, manually hide/show my layers in order to explore my data. Quite tiring when you have a lot of data.

Now, let’s see what the new Galigeo for Lumira version is giving me back when I configure my Indicator and my Geolayers dropboxes the same way.

Interesting enough indeed. This time, only the top layer is displayed, and there is an additional icon on the top left of the map.

This icon allows me perform two actions : drill down and drill up.

Let’s try them right now.

I click on the drill down button. Now when hovering a sector on the map, it’s outlined in red.

Let’s click on the sector which has the best turnover potential.

Magic happened ?

The sector layer has been drilled down and now only the subsectors of the drilled sector are displayed.

Drill down being still enabled, I’ll click again on the subsector which has the highest potential turnover.

Same effect, the subsector has been drilled down, and now only the pubs/restaurants that geographically belong to the drilled subsector are being shown.

The bubbles representation is giving me a a clear idea of which pubs/restaurants have the most potential turnover.

It’s gonna be hot this summer, better send the Sales Reps to visit these pubs and restaurants!

The drill up button has, as you might  expect, the inverted effect. If I click on it, it will drill up from the pubs/restaurants layer to the subsectors layer.

You might notice that the filter on subsectors layer, that was made when drilling down from sectors to subsectors, is still active, which is nice to keep on exploring other subsectors from the initially selected sector.

That’s pretty much it.

Yeah, it’s that simple.

No configuration needed, that drill hierarchy can take as many layers as you want.

For example, it could be Country > Regions > SubRegions > Custom Subregion > POI and it would still work.

The few prerequisites, which are coming from common sense, are :

• The first layer of the hierarchy has to display polygon
• At least two layers must share the same indicator (but by experience, really often they all share the same indicator)

Really excited or still not convinced?

The best thing you can do then is to download the Galigeo for Lumira extension (if it’s not already done), and start playing with it.

As always, feedbacks or criticisms or even ideas you might have are always welcome.

Also note that Galigeo for Lumira is coming soon for Lumira Server for BI Platform.

Cheers,

Vincent

Big data has been the Buzz all around; more applications are being built on top of Big data platform and the whole paradigm shift takes a move towards NoSQL Databases from traditional RDBMS. When these databases start to hold transactional and historical data, there is a need for reporting. SAP Lumira comes as a handy tool as it focuses on Big data and is continuously improving capabilities in handling them.

SAP Lumira 1.27 brought in better integration with Big data like Hadoop. You can refer to my blogs on how to connect your SAP Lumira to MongoDB here: http://scn.sap.com/community/lumira/blog/2014/06/17/connecting-sap-lumira-with-mongodb--part-1 and http://scn.sap.com/community/lumira/blog/2014/06/17/connecting-sap-lumira-with-mongodb--part-2. In this blog, we will describe how to connect SAP Lumira to another growing NoSQL database called RethinkDB using Custom Data Access Extension.

RethinkDB :

RethinkDB is fairly new and improving, with features like Real time notification, easy to setup clusters and improved ReQL language. The ability to pull JSON data directly from web makes RethinkDB an ideal choice for next generation web applications.

As a step to demonstrate the Custom Data Access Extension, we will import example data “Election Analysis” from their website. The JSON data is imported into RethinkDB into the test database (you can refer the tutorial http://www.rethinkdb.com/docs/tutorials/elections/ to do the same). The Custom Data Access extension that I had built to connect to RethinkDB is installed in SAP Lumira now. Find more about how to configure your SAP Lumira for Data Access Extension and how to build one at Lumira - Open Source Data Access Extensions

Getting data from RethinkDB

RethinkDB Extension appears under the External Datasource in the Dataset Pane in SAP Lumira. Select the Rethink External Data source and click next to progress.

A new window appears asking information about the RethinkDB instances to connect and fetch the data. Enter the following information:

• host where Rethinkdb is installed
• port on which it’s running (leaving blank defaults to port 28015)
• db to connect to (leaving blank lists all the db that is present in the instance)
• authkey

Since I left db as blank, it shows me list of all databases in the system. Select the required database and click Ok. In this case, I am selecting ‘test’ as that is where the example data is loaded.

The next screen shows list of tables. Select the required table and click Ok. Here, I am selecting ‘polls’ that contains the cleansed data from the example imported.

You get the data set preview of what is going to be imported. Click Create to import the data.

You can manipulate this data like any other regular data in the prepare tab in SAP Lumira. You can also import another table by following the above steps and merging them in SAP Lumira.

Just a Visual

The visualization below was made using the example data. You can refer to the example link above for information about the data.

Top 5 states by poll

Gross Operating Profit for each state

Top 10 Democrats

The Custom Data Access Extension is easy to use and you can now do analysis on top of RethinkDB using SAP Lumira. You can download the extension from https://github.com/sgsshankar/lumira-extension-da-rethinkdb

In the future version of Custom Data Access Extension, support for live steaming, map reduce capabilities and other rethinkDB query functionalities would be added to extend the full potential of RethinkDB to Visualize on SAP Lumira.

Like this blog! Want to Scrape live website and consume data into Lumira? Now its possible! Check out Website to Visuals – Scraping Website into Data within SAP Lumira

Love this Blog! Tip this blog @ 1DtfHaPcUCaA95WY4eaM2se7kkvPQFELsz

Here is my gift to you. Don’t spend time trying to create the optimal schedule of BI sessions for SAP Analytics and BusinessObjects Conference (SABOC) 2015 in Austin (August 31 – Sept 2). I’ve taken care of that for you. Here is my curated list of really great educational sessions, according to these criteria:

• Anything longer than 90 minutes is too long without caffeine.
• Customer sessions get preference.
• Roadmap sessions can also be heard on SAP webcasts, so they got lower preference.

Don't miss the opportunity to investigate our new Data Genius contest, too.

I look forward to seeing you there! You can always follow the action at @InaSAP, @ASUG_BI, or @SAPAnalytics.

Gartner BI & Analytics Magic Quadrant 2016: Participants Needed

As part of their yearly research in the BI and Analytics market, industry analyst firm Gartner will be conducting a survey of users of BI and Analytics platforms in early Q4 2015. The survey is expected to collect information in the areas of tool functional strengths and gaps, as well as vendor’s support and sales processes. It will support the next issue of the Magic Quadrant for BI and Analytics Platforms to be published in 2016. As you know Gartner Magic Quadrant reports are often used by companies to better understand the market and the vendors they are evaluating for their business.

Your participation to this 20-30 minute web-based survey will provide valuable insights to Gartner on how SAP Lumira is being applied to drive further innovation and vision for the market. The data collected will be kept strictly confidential. This is not a request for public endorsement. In appreciation of participation, Gartner will send you copies of MQ research notes highlighting data from the survey.

If you are interested in participating or if you just need more information about this Gartner survey, please sign up online here, an SAP product expert will contact you to confirm your participation or address your need for more information. You may also contact Thierry AUDAS Sr Director, BI Product Marketing, by email (thierry.audas@sap.com).

Your participation and feedback to Gartner is very important to SAP, we appreciate your contribution.

Thank you!

*: Current SAP Lumira software includes:

• Standard edition (for desktop users)
• Server for teams (licensed with the Edge edition)
• Server for the SAP BusinessObjects BI Platform (licensed with the Edge edition or the SAP BI Suite)
• Cloud edition

Cycling is the best option to commute when short or moderate distance is into consideration. Specifically when in cities, biking is economic and effective mode of transit.

Bicycles provide numerous benefits in comparison with motor vehicles, including the sustained physical exercise involved in cycling, cycling involves a reduced consumption of fossil fuels, less air or noise pollution, much reduced traffic congestion, easier parking, increased maneuverability, and access to both roads, paths and rural trails. The advantages also include reduced financial cost to the user as well as to society at large.

Because of its benefits I have always admired biking to work and feel amazed when people commute to office using bicycles instead of commonly used automobiles.(by Biking I refer to Bicycling in this blog)

1. Unfortunately in my case biking to office is not a possibility as I stay 20 miles away from the place I work. But by this it doesn’t mean that I don’t bike at all, yes I have weekends to unleash this adventurous hobby. I bike as much as possible every weekend to stay fit and healthy J

Before I begin my blog I encourage as many readers as possible to take up Bike to Work as a hobby and start biking ASAP!

In this blog I have used SAP Lumira to enliven the commuting data extracted from below sources

1. The Census Bureau – American community Survey 2013
2. The League of American Bicyclist

The Report is specific to United States of America and has captured various statistics of American Bicycle commuting trends for various years.

Limitation: SAP Lumira couldn’t detect cities like Davis, California and Boulder, Colorado hence could not represent them on Geo Charts. (However, managed to represent them on bar and other chartsJ)

• The visuals and data can be useful to bicycle manufacturers to target their product in the US geography.
• An insight to aspiring bikersJ
• An urge to increase bike commuting percentage in the US and other Countries.

Notable points:

1. Gathering Bicycle commuting data was indeed a task.
2. Cleansing data to make it ready for LUMIRA feed, even more time consuming task.
3. Use of Geo Maps, Trellis and Custom Hierarchy functionality in SAP LUMIRA was of great help.
4. Was great fun to work on Network Chart

Hope this will be a visual treat to the reader. Thanks to SAP LUMIRA!

State wise Break:

Top states with highest % change of bike commuters from 2005 to 2012

Statewise Commute by Bike in 2012 and 2005 as a percentage of Total Population

Top states with highest % of bikers wrt to Total Population

Largest City (Population wise) and Bike Commute Percentage

Largest Cities in US (Population 200,000 – 2 million +) – Ranked on the basis of Bike Commuters as % of Total population of the City

Heat Map plotting State and City

Which state and its cities has cycling popularity ?

Fastest growing Cities in terms of biking population over the decade 1990 to 2013

Classification of country in to Regionsexplained with the help of Tree and Network charts

Total Population and Biking population for each region

Region wise classification of Biking/Non Biking/Total Population using Trellis

US City Size(Population) classification explained with the help of Network and Tree Charts

Bike Commuters classified based on City Population with the help of Geo Map Trellis

Pie chart trellis:

Color represents City

Pie Section 1^st row – No of Bike commuter

Pie Section 2^nd row - Population

Mode of transit share in the US

Mode of transit share in each state:

Which city do people prefer to walk or Bike?

Most number of Bicyclist in a City in 2013

No. of Bicyclist in a city mapped against total population of that city

Commuter percentage, No. of Bicyclist & Total Population on Parallel Co-ordinates chart

Which city has the highest number of Bicyclist in the US?

Which City has the highest percentage of Total Population commuting by Bike?

Some intersting figures

Before I end my blog some facts for the year 2013

Bitcoins is gaining popularity among the masses, if you are a crypto fan or have not heard bitcoins before, then you should definitely check it out. Bitcoin is a form of Crypto Virtual currency that you can use pretty much like a real world currency, except its Virtual. I have been following bitcoins for long time and I am going to use SAP Lumira to build simple storyboard that help to analyse on latest bitcoin stats.

Since it’s going to be live data, I am going to use a NoSQL database designed to handle realtime web data, it’s called RethinkDB and you can refer my blog Rethink your Big Data with SAP Lumira and Rethi... | SCNon how to consume data from RethinkDB in SAP Lumira. Let’s proceed on creating our storyboard!!

Prepare the Data

Data on bitcoins are present pretty much everywhere on the internet. Realtime API’s and stats data can be collected using web service. I have sourced some data from https://data.bitcoinity.org/ and https://blockexplorer.com/q through real time application that writes the data onto RethinkDB. All the operations of merging and map reduce are done at RethinkDB so that only the necessary data is brought into Lumira.

I am using the lumira-rethinkdb Data Access extension to connect to my RethinkDB and bring the data into Lumira. I have a Database called bitcoin and inside that various tables holding various data like price, trade and the blockchain information.

All the tables are linked by foreign key Time using which different tables’ data are merged inside SAP Lumira.

I am merging data of all the tables present and finally I get the complete dataset in my Prepare tab in SAP Lumira

We have now obtained the data. Let’s visualize the data to understand it even better.

Visualize

We are using Time as our main Dimension since its real time data. We will be using different measures to analyze different data related to bitcoins and its block chains.

Bitcoin is slowly adapting into a form of regular currency. As the web gets Realtime, we would also need Realtime Visualization and use of NoSQL database to support the flow of data. SAP Lumira works very well with databases like RethinkDB and MongoDB which you can see here https://scn.sap.com/community/lumira/blog/2014/06/17/connecting-sap-lumira-with-mongodb--part-1 and https://scn.sap.com/community/lumira/blog/2014/06/17/connecting-sap-lumira-with-mongodb--part-2

Love this Blog! Tip this blog @ 1DtfHaPcUCaA95WY4eaM2se7kkvPQFELsz

In the past year or so, I have presented at various occasions on how to create visualization extensions for SAP Lumira, and all the examples I demonstrated so far were based on D3. Needless to say, D3 is extremely powerful and flexible, but it does not come with built-in charts, so we often have to create a chart from scratch, which could be a daunting task, especially if you want to create an extension that can accommodate various data formats/volume for productive use.  This can be even more difficult for someone who is not familiar with D3.

But why do we have to re-invent the wheel if somebody has already created such a chart? I am working on a presentation for the upcoming SABOC conference in Austin (Aug 31 - Sep 2) to demonstrate the use of a high-level charting library that contains built-in charts. The example that I am going to show is a Sankey Diagram based on Google Visualization API. Within a few lines of code, a complex Sankey Diagram visualization is created, and it can handle any number of source/intermediate/target nodes without any extra code. The charting library has done all the heavy lifting for us, and we simply just have to instantiate and parameterize the chart.

So if you are interested, come and attend my SABOC session: Create Cool Lumira Visualization Extensions with SAP Web IDE. If you can't wait and would like to see a sneak preview of the extension, feel free to find it on the Github repository of Lumira visualization extensions. Here is a screenshot of the Sankey Diagram that visualizes British Columbia's provincial budget spending.

Another Sankey Diagram that visualizes the energy consumption flow:

See you at the conference!

Lumira shops are expanding data discovery to cloud data, and it often starts in sales and marketing operations. Sales is finding new insight through analytics across opportunities, pipelines and other custom objects in Salesforce, MS Dynamics, Oracle Service Cloud or SugarCRM.  For marketing operations, we're seeing data driven teams run analysis on campaign and activity data across Eloqua, Marketo, Hubspot and Google Analytics.

These connections are being established using the DataDirect Cloud BI service (JDBC, OData, and ODBC) which is built by the same engineering expertise that developed connectors for SAP BI enterprise reporting and warehousing tools.  DataDirect Cloud is self service and connects to popular data sources across SaaS applications, hosted databases and firewall friendly connectivity over HTTPS.  Regarding connecting over HTTPS, DataDirect Cloud can even get to data sources behind your firewall from the cloud so submit an idea here if you'd like to see that capability in SAP Lumira Cloud.

Getting Started with SAP Lumira and Oracle Eloqua (same steps apply for any supported data source):

1. Sign up for DataDirect Cloud Trial

2. The first step is to create a Data Source definition. On the Dashboard, you can click on ‘Data Sources’ tab to the left of Dashboard and then you would have to click on ‘Data Stores’ on the Next Screen.

3. Select Eloqua for this tutorial and enter your connection information.

Note: This is usually sent via e-mail by your Eloqua Admin if you do not already have it.

4. Test the connection to make sure you are getting connected successfully.

5. On the Left side, you will find a link Downloads, from where you can download DataDirect Cloud JDBC Drivers and install them.

Note: DataDirect Cloud provides a SQL compliant interface to all of the supported data sources via JDBC.  In addition, those same sources can produce OData which we will discuss in future tutorial.

6. Open SAP Lumira and Add New Data Sheet.

7. In the Add New Dataset form select your source as  ‘Query With SQL’  and press Next.

8. On the Next Screen, you will be shown all the available Databases. Check the Checkbox ‘Show all’ on right side to view other sources of databases that are unavailable. Now head over to Generic drivers and choose "Generic JDBC Datasource - JDBC Drivers"

9. Install the Generic JDBC drivers by clicking on Install button. You would have to browse to "C:\Program Files\Progress\DataDirect\Cloud_for_JDBC_20\lib\ddcloud.jar" and press ok to install the drivers and press Next.

10. On the Next Screen, fill in your DataDirect Cloud credentials and JDBCURL and JDBC Class.

JDBCURL=jdbc:datadirect:ddcloud://service.datadirectcloud.com;databaseName=<Data Source Name>

<Data Source Name>: It’s the Name of your datasource that you have configured to connect to Eloqua in DataDirect Cloud.

JDBCClass = com.ddtek.jdbc.ddcloud.DDCloudDriver

11. Click on Connect and on the Next screen you will find all your tables and you can select which data that you need to visualize by running a SQL query. Below is a basic visualization using the above setup showing the number of Emails sent by Account.

Troubleshooting:

On SAP Lumira, when you try to connect to DataDirect Cloud for the first time, If may encounter the following error:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

To resolve, follow steps in the following SAP knowledge article:

2113614 - How do I set up SSL for SAP Lumira connecting to SAP Lumira Server?

• Get the InstallCert program from here.
• Compile the program and run it to retrieve the certificate from server as below:

java InstallCert service.datadirectcloud.com

• Extract certificate from created jssecacerts keystore using the following command.

keytool -exportcert -alias service.datadirectcloud.com-1 -keystore jssecacerts -storepass changeit -file service.datadirectcloud.com.cer

• Import certificate into keystore that came with SAP Lumira at the location C:\Program Files\SAP Lumira\Desktop\jre\lib\security using command:

keytool -importcert -alias service.datadirectcloud.com-   keystore C:\Program Files\SAP Lumira\Desktop\jre\lib\security -storepass changeit -file service.datadirectcloud.com.cer

About the author:

Sumit Sarkar, Chief Data Evangelist

Progress DataDirect

@SAsInSumit

www.linkedin.com/in/meetsumit

Hi, this is my first blog entry in the Lumira space (No, I'm not lost, I promise!  )

I've spend most of my time over in SAP BusinessObjects Design Studio space but once in a while I get a wild hair to try something different.  As those who follow me in the Design Studio space know, we have a pretty big Open Source repository available for Design Studio extensions.  (SCN Design Studio SDK Development Community)  Our GitHub repo is here: org-scn-design-studio-community/sdkpackage · GitHub -- While our page and GitHub repository isn't always as pretty/tidy as what I've seen over in Lumira's equivalent page and repo, I have some visualization components that I thought would be a good fit to try to port over to Lumira so here I am. 

The component I chose to use is a Hexagonal Binning component that I developed a while back, and posted a recent blog entry in Design Studio that you can see here (Design Studio SDK - Hexagonal Binning) and this is an adaptation for it that is meant for Lumira, but not necessarily a copy and paste verbatim with that entry.

So what is it?  A HexBin Chart allows us to visualize x/y data in a bit of a different way.  Traditional Scatter Plots and Bubble Charts are good at showing correlations (or lack thereof) such as below:

A Scatter Plot showing Hits (x-axis) compared to unique visitors (y-axis).

The example above is a manageable visualization because of the low volume of data points where the legend and colors are identifiable and can for the most part be distinguished from one another.

For larger data points, this may not be as useful:

A Scatter Plot showing 2000 data points.  Or a pile of marbles.

This visualization doesn't seem to be as useful.  There's no indication if there's a certain area of more plot density than another.  Generally, the data points seem to be within a range of 0 to 500 and 200 to 700, but how uniform is the distribution?

I took a look at some of Mike Bostock's examples and liked what I saw with his HexBin example here: Hexagonal Binning

Let's look at the HexBin's representation of this same data:

A HexBin using 'Median' sampling to set color binning.

As you can see above, we can use hexagonal binning to start binning certain thresholds of density into a color scale (dictated in color and occurrence by your color palette).  I have 3 automated options to try to bin them.  Above was using Median, and below we see two others.  Depending on the profile of your data, one may work better than another:

Mean and Max methods shown.  Same data, but note the difference in the color distribution thresholds in the legend.

Next, suppose you need a little more fidelity or less fidelity in your hexagon bins.  I have a radius parameter that you can control how large the hexagons are.  Below are examples of different sizes and colors for sake of variety:

A larger hexagon size with a multi-hue color range.

A smaller hexagon size with simpler color range

Lessons Learned/Comparisons with Design Studio:

• Lumira seems to have a more fully implemented RequireJS dependency pattern which took me some getting used to.
• I was unable to expose charting parameters (as shown in the Compose Room screenshots above) in the Visualize Room.  Ideally, I'd like to have them available there in the Context Menu (the little Gear icon) when I could have a dialog box come up to tweak these properties.
• In Design Studio, I provide a Legend also, but did not implement in Lumira because I'm exploring the possibility of using their Flow Layout convention to render it there which is different than the Design Studio approach.
• Aside from that limitation, I was able to re-use some of my Design Studio Property Editor components such as the palette builder to save Property Editor development time.  Below is an illustration of the Design Studio equivalent, for example:
  
  
  Design Studio Equivalent showing Advanced Property Sheet and Legend.
  

Feedback is welcomed.  You can view the source and download the .ZIP bundle here (which includes more goodies):

entmike/lumira · GitHub*

Enjoy!

*PS: This repository has a ZIP file in the root that you can simply install and the repository itself is one big Eclipse project as I developed this largely without WebIDE or VizPacker after getting some template code, but I'll save those gory details for another blog post.  Also I'm interested in contributing to SAP/lumira-extension-viz · GitHub after cleaning up a few pieces, but I'm not sure who to reach out to to proceed yet.

Coming directly off my making SAP Lumira Visualization Extension - Hexagonal Binning, I wanted to try porting over another D3 Example by Mike Bostock that I've always liked.

(Scatterplot Matrix Brushing and http://mbostock.github.io/d3/talk/20111116/iris-splom.html)

I liked this visualization, because it seemed to me that you could add any amount of additional measures and the visualization could technically just add additional rows and columns to the matrix.  The brush/lassoing is a nice effect to highlight where in the correlation the data points fall in other correlations.  I didn't really like the diagonal intersection of cells personally, but I can see how perhaps one could get a sense of which category occurs where, almost like a spectrometer in some ways...

So anyway - I set off on porting this one over to support any number of Measures, and making the first Dimension that you pull in dictate the plot color.  Below are a few sample screenshots:

3 Measures

5 Measures

7 Measures (My computer fan started going crazy here so this I think is overkill )

If you draw your attention to the diagonal items, I've replaced those with histograms based on another D3 example here: Histogram

Overall, I think this visualization is a helpful data profiling and data discovery tool to identify correlations to then do something else with.

Feedback is welcomed.  You can view the source and download the .ZIP bundle here (which includes more goodies):

entmike/lumira · GitHub*

Enjoy!

*PS: This repository has a ZIP file in the root that you can simply install and the repository itself is one big Eclipse project as I developed this largely without WebIDE or VizPacker after getting some template code, but I'll save those gory details for another blog post.  Also I'm interested in contributing to SAP/lumira-extension-viz · GitHub after cleaning up a few pieces, but I'm not sure who to reach out to to proceed yet.