{ "version": "https:\/\/jsonfeed.org\/version\/1", "title": "LEFT JOIN: blog on analytics, visualisation & data science, posts tagged: visualisation", "home_page_url": "https:\/\/en.leftjoin.ru\/tags\/visualisation\/", "feed_url": "https:\/\/en.leftjoin.ru\/tags\/visualisation\/json\/", "icon": "https:\/\/en.leftjoin.ru\/user\/userpic@2x.jpg", "author": { "name": "Nikolay Valiotti", "url": "https:\/\/en.leftjoin.ru\/", "avatar": "https:\/\/en.leftjoin.ru\/user\/userpic@2x.jpg" }, "items": [ { "id": "69", "url": "https:\/\/en.leftjoin.ru\/all\/bubble-charts-basics-area-vs-radius\/", "title": "Bubble charts basics: area vs radius", "content_html": "

Data visualization is a skill used in any industry where data is present, because tables are only good for storing information. When there is a need to present data, or rather certain conclusions derived from them, the data must be presented on graphs of a suitable type. So, here you are faced with two tasks: the first is to choose the right type of graph, the second is to display the results in a plausible way. Today we will tell you about one mistake that designers sometimes make when visualizing data on bubble-charts and how this mistake can be avoided.<\/p>\n

The crux of building a bubble-chart<\/h2>\n

Firstly, let us tell you a bit of boring theory before we start analyzing the data. Bubble-chart is a convenient way to show three numerical variables without building a 3D model. The usual X and Y axes indicate the values ​​of two parameters, and the third is shown by the size of the circle that corresponds to each observation. This is what makes it possible to avoid the need to build a complex 3D chart, that is, anyone who sees a bubble-chart will be able to draw conclusions about the data much faster.<\/p>\n

A mistake that a designer, but not a data analyst, can make<\/h2>\n

With the metrics that are displayed on the axes of the graph, no questions arise. This is the usual way of visualizing data, but with the data shown by the size of the circles there is some difficulty: how to correctly and accurately display changes in the values ​​of a variable, if the control is not a point on the axis, but the size of this point?
\nThe fact is that when building such a graph without using analytical tools, for example, in a graphics editor, the author can draw circles, taking the radius of the circle as its size. At first glance, everything seems to be absolutely correct – the larger the value of the variable, the larger the radius of the circle. However, in this case, the area of ​​the circle will increase not as a linear, but as a power function, because S = π × r2. For instance, the figure below shows that if you double the radius of a circle, the area will quadruple.<\/p>\n


\n Draw a circle in Matplotlib <\/span> <\/summary><\/p>\n
fig = plt.figure (figsize = (10, 10))\r\nax = fig.add_subplot (1, 1, 1)\r\ns = 4 * 10e3\r\n\r\n\r\nax.scatter (100, 100, s = s, c = 'r')\r\nax.scatter (100, 100, s = s \/ 4, c = 'b')\r\nax.scatter (100, 100, s = 10, c = 'g')\r\nplt.axvline (99, c = 'black')\r\nplt.axvline (101, c = 'black')\r\nplt.axvline (98, c = 'black')\r\nplt.axvline (102, c = 'black')\r\n\r\n\r\nax.set_xticks (np.arange (95, 106, 1))\r\nax.grid (alpha = 1)\r\n\r\nplt.show ()<\/code><\/pre>
<\/details><\/p>\n
\n\"\"\n<\/div>\n
This means that the graph will look implausible, because the dimensions will not reflect the real change in the variable, and the viewer pays attention and compares exactly the area of ​​the circles on the graph.<\/p>\n
How to build such a graph correctly?<\/h2>\n
Fortunately, if you build bubble-charts using Python libraries (Matplotlib and Seaborn), then the size of the circle will be determined by the area, which is absolutely correct from in terms of visualization.
\nNow, using the example of real data found on Kaggle, we will show how to build a bubble-chart. The data contains the following variables: country, population size, percentage of literate population. For the chart to be readable, let’s take a subsample of the top 10 countries after sorting all the data in order of increasing GDP.<\/p>\n
First, let’s load all the necessary libraries:<\/p>\n
import pandas as pd\r\nimport matplotlib.pyplot as plt\r\nimport seaborn as sns<\/code><\/pre>
Then, load the data, clear it from all rows with missing values ​​and transform the population of countries to millions of people:<\/p>\n
data = pd.read_csv ('countries of the world.csv', sep = ',')\r\ndata = data.dropna ()\r\ndata = data.sort_values ​​(by = 'Population', ascending = False)\r\ndata = data.head (10)\r\ndata ['Population'] = data ['Population']. apply (lambda x: x \/ 1000000)<\/code><\/pre>
Now that all the preparations are complete, you can build a bubble-chart:<\/p>\n
sns.set (style = "darkgrid")\r\nfig, ax = plt.subplots (figsize = (10, 10))\r\ng = sns.scatterplot (data = data, x = "Literacy (%)", y = "GDP ($ per capita)", size = "Population", sizes = (10,1500), alpha = 0.5)\r\nplt.xlabel ("Literacy (Percentage of literate citizens)")\r\nplt.ylabel ("GDP per Capita")\r\nplt.title ('Chart with bubbles as area', fontdict = {'fontsize': 'x-large'})\r\n\r\ndef label_point (x, y, val, ax):\r\n    a = pd.concat ({'x': x, 'y': y, 'val': val}, axis = 1)\r\n    for i, point in a.iterrows ():\r\n        ax.text (point ['x'], point ['y'] + 500, str (point ['val']))\r\n\r\nlabel_point (data ['Literacy (%)'], data ['GDP ($ per capita)'], data ['Country'], plt.gca ())\r\n\r\nax.legend (loc = 'upper left', fontsize = 'medium', title = 'Population (in mln)', title_fontsize = 'large', labelspacing = 1)\r\n\r\nplt.show ()<\/code><\/pre>
\n\"\"\n<\/div>\n
This graph displays three metrics in an understandable way: the level of GDP per capita on the Y axis, the percentage of the literate population on the X axis, and the population – by the area of ​​the circle.<\/p>\n
We recommend using size of the circle to show one of the variables, if there is a need to show 3 or more variables on one chart.<\/p>\n",
            "date_published": "2021-10-14T17:58:24+03:00",
            "date_modified": "2021-10-14T17:58:05+03:00",
            "image": "https:\/\/en.leftjoin.ru\/pictures\/example.png",
            "_date_published_rfc2822": "Thu, 14 Oct 2021 17:58:24 +0300",
            "_rss_guid_is_permalink": "false",
            "_rss_guid": "69",
            "_e2_data": {
                "is_favourite": false,
                "links_required": [
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css",
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css",
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css",
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css"
                ],
                "og_images": [
                    "https:\/\/en.leftjoin.ru\/pictures\/example.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/_32.png"
                ]
            }
        },
        {
            "id": "39",
            "url": "https:\/\/en.leftjoin.ru\/all\/pandas-profiling-in-action\/",
            "title": "Pandas Profiling in action:  reviewing a new EDA library on Superstore Sales dataset",
            "content_html": "
Before moving directly to data analysis we need to understand what type of data we are going to work with. In today’s material, we will take a closer look at the SuperStore Sales dataset, specifically at the Orders<\/i> column. It includes customer shopping data of a Canadian online supermarket, such as order, product and customer ids,  type of shipping, prices, product categories, names and etc. You can find more information about this dataset on GitHub<\/a>. After creating a pandas DataFrame we can simply  use the describe()<\/span> method to get a sense of our data.<\/p>\n
import pandas as pd\r\n\r\ndf = pd.read_csv('superstore_sales_orders.csv', decimal=',')\r\ndf.describe(include='all')<\/code><\/pre>
And oftentimes it leads to such a mess:<\/p>\n
\n\"\"\n<\/div>\n
The source code of this library is available on GitHub<\/a><\/p>\n
If we spend some time trying to get a grasp of this descriptive table,  we can find out that customers are more likely to choose “Regular air” as a shipping type or that the majority of orders were made from Ontario.  Nevertheless, there is a better tool to describe the dataset in more detail  – the pandas-profiling library.  Just pass a DataFrame to it and we will get a generated HTML page with a detailed description of our dataset:<\/p>\n
import pandas_profiling\r\nprofile = pandas_profiling.ProfileReport(df)\r\nprofile.to_file("output.html")<\/code><\/pre>
\n\"\"\n<\/div>\n
As you see, it returned a page with 6 sections, namely: overview, variables, interactions and correlations, number of missing values, and dataset samples.<\/p>\n
View a full version of the Pandas Profiling Report<\/a><\/p>\n
Data overview<\/h2>\n
Let’s move to the first subsection called “Overview”.  Pandas Profiling provided the following stats: number of variables, number of observations, missing cells, duplicates, and file size. The  Variable types<\/span>  column shows that our DataFrame consists of 12 categorical and 9 numerical variables.<\/p>\n
\n\"\"\n<\/div>\n
The  “Reproduction”  subsection stores technical information,  showing how long it took to analyze the dataset,  currently installed version , configuration info and etc.<\/p>\n
\n\"\"\n<\/div>\n
The  “Warnings”  subsection informs about possible issues in the dataset structure. Now,  it warns us that the “Order Date” column has too many distinct values.<\/p>\n
\n\"\"\n<\/div>\n
Variables<\/h2>\n
Moving further, this subsection contains a detailed description of each variable, displaying the number of duplicates and missing values stored, memory size, maximum and minimal values. Right next to the stats you can see the distribution of column values.<\/p>\n
\n\"\"\n<\/div>\n
Clicking on  Toggle details<\/span>  you will see more expanded information:  quartiles, median and other useful descriptive statistical indicators. The remaining tabs contain a histogram displayed on the main screen, top 10 frequent values and extremes.<\/p>\n
\n\"\"\n<\/div>\n
Interactions<\/h2>\n
This section displays how variables are interconnected on a hexbin plot: The graph looks not very obvious and clear, since the legend is lacking.<\/p>\n
\n\"\"\n<\/div>\n
Correlations<\/h2>\n
The section represents correlations between variables calculated in a variety of ways. For example, the first tab shows Pearson’s r-value. It is noticeable that Profit <\/span> is positively correlated with  Sales<\/span>.  You can get a detailed explanation to each coefficient by clicking on the Toggle correlation descriptions<\/span> button.<\/p>\n
\n\"\"\n<\/div>\n
Missing values<\/h2>\n
This section includes a bar chart, matrix, and dendrogram with the number of fields in each variable. For instance,  the  Product Base Margin<\/span>  column is missing three values.<\/p>\n
\n\"\"\n<\/div>\n
Samples<\/h2>\n
And the final section show the first and last 10 rows as chunks of a dataset, pretty similar to the  head()<\/span>  method in Pandas.<\/p>\n
\n\"\"\n<\/div>\n
Key Takeaways<\/h2>\n
The library is definitely more focused on statistics than Pandas, one can get useful descriptive stats for each variable and see their correlation.  It provides a comprehensive report on a dataset in a user-friendly way,  allowing to undertake an initial investigation and get a sense of data.
\nStill, the library has its shortfalls. If your dataset is fairly large the report generation time may be extended up to several hours. It’s a great tool for automating EDA tasks,  however, it can’t do all the work for you and some details may be overlooked. If you are just getting started with data analysis, we would highly recommend to start it with pandas. It will solidify your knowledge and boost confidence in working with data.<\/p>\n",
            "date_published": "2020-09-18T15:37:40+03:00",
            "date_modified": "2020-09-18T15:42:43+03:00",
            "image": "https:\/\/en.leftjoin.ru\/pictures\/1-15.png",
            "_date_published_rfc2822": "Fri, 18 Sep 2020 15:37:40 +0300",
            "_rss_guid_is_permalink": "false",
            "_rss_guid": "39",
            "_e2_data": {
                "is_favourite": false,
                "links_required": [
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css",
                    "system\/library\/highlight\/highlight.js",
                    "system\/library\/highlight\/highlight.css"
                ],
                "og_images": [
                    "https:\/\/en.leftjoin.ru\/pictures\/1-15.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/2-15.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/3-14.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/4-12.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/5_5.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/6_6.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/7_7.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/8_8.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/9_9.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/10_10.png",
                    "https:\/\/en.leftjoin.ru\/pictures\/11_11.png"
                ]
            }
        },
        {
            "id": "34",
            "url": "https:\/\/en.leftjoin.ru\/all\/building-an-interactive-waterfall-chart-in-python\/",
            "title": "Building an interactive waterfall chart in Python",
            "content_html": "
Back in 2014, we built a waterfall chart in Excel, widely known in the consulting world, for one of our presentations about the e-commerce market in Ulmart. It’s been a while and today we are going to draw one in Python and the Plotly library<\/a>. This type of charts is oftentimes used to illustrate changes with the appearance of a new positive or negative factor. In the latter article about data visualization, we explained how to build a beautiful Bar Chart with bars that resemble thermometers<\/a>, it’s especially useful when we want to compare planned targets with actual values.<\/p>\n
\n\"\"\n<\/div>\n
We are using the Ulmart data on the e-commerce market growth from 2013 to 2014. Data on the X-axis is chart captions,  on the Y-axis we displayed the initial and final values,  as well as their change. With the sum() function calculate the total and add it to the end of our list. The <br> tag in the x_list<\/span>  shows a line break in text.<\/p>\n
import plotly.graph_objects as go\r\n\r\nx_list = ['2013','The Russian <br>Macroeconomy', 'Decline in working age<br>population','Internet usage growth','Development of<br>cross-border trade', 'National companies', '2014']\r\ny_list = [738.5, 48.7, -7.4, 68.7, 99.7, 48.0]\r\ntotal = round(sum(y_list))\r\ny_list.append(total)<\/code><\/pre>
Let’s create a list with column values, we called it  text_list<\/span>. The values will be taken from the y_list<\/span>,  but first we need to transform them. Convert all numerical values into strings and if it’s not the first or the last column, add a plus sign for clarity. In case it’s a positive change, the color will be green, otherwise red. Highlight the first and the last values with the <b> tag;<\/p>\n
text_list = []\r\nfor index, item in enumerate(y_list):\r\n    if item > 0 and index != 0 and index != len(y_list) - 1:\r\n        text_list.append(f'+{str(y_list[index])}')\r\n    else:\r\n        text_list.append(str(y_list[index]))\r\nfor index, item in enumerate(text_list):\r\n    if item[0] == '+' and index != 0 and index != len(text_list) - 1:\r\n        text_list[index] = '<span style="color:#2ca02c">' + text_list[index] + '<\/span>'\r\n    elif item[0] == '-' and index != 0 and index != len(text_list) - 1:\r\n        text_list[index] = '<span style="color:#d62728">' + text_list[index] + '<\/span>'\r\n    if index == 0 or index == len(text_list) - 1:\r\n        text_list[index] = '<b>' + text_list[index] + '<\/b>'<\/code><\/pre>
Let’s set parameters for the dashed lines we want to add. Create a list of dictionaries and fill it with light-gray dashed lines, passing the following:<\/p>\n
dict_list = []\r\nfor i in range(0, 1200, 200):\r\n    dict_list.append(dict(\r\n            type="line",\r\n            line=dict(\r\n                 color="#666666",\r\n                 dash="dot"\r\n            ),\r\n            x0=-0.5,\r\n            y0=i,\r\n            x1=6,\r\n            y1=i,\r\n            line_width=1,\r\n            layer="below"))<\/code><\/pre>
Now, create a graph object with the Waterfall()<\/span> method. Each column in our table can be of a certain type: total<\/span>, absolute<\/span> (both with final values) or relative<\/span> (holds intermediate values). Then we need to set colors, make the connecting line transparent, positive changes will be green, while negative ones are red, and the final columns are purple. Here we are using the Open Sans font.<\/p>\n
Learn more about how to choose the right fonts for your data visualization from this article: “Choosing Fonts for Your Data Visualization”<\/a><\/p>\n
fig = go.Figure(go.Waterfall(\r\n    name = "e-commerce", orientation = "v",\r\n    measure = ["absolute", "relative", "relative", "relative", "relative", "relative", "total"],\r\n    x = x_list,\r\n    y = y_list,\r\n    text = text_list,\r\n    textposition = "outside",\r\n    connector = {"line":{"color":'rgba(0,0,0,0)'}},\r\n    increasing = {"marker":{"color":"#2ca02c"}},\r\n    decreasing = {"marker":{"color":"#d62728"}},\r\n    totals={'marker':{"color":"#9467bd"}},\r\n    textfont={"family":"Open Sans, light",\r\n              "color":"black"\r\n             }\r\n))<\/code><\/pre>
Finally, add the title with the description, hide the legend, set the Y label and add dashed lines to our chart.<\/p>\n
fig.update_layout(\r\n    title = \r\n        {'text':'<b>Waterfall chart<\/b><br><span style="color:#666666">E-commerce market growth from 2013 to 2014<\/span>'},\r\n    showlegend = False,\r\n    height=650,\r\n    font={\r\n        'family':'Open Sans, light',\r\n        'color':'black',\r\n        'size':14\r\n    },\r\n    plot_bgcolor='rgba(0,0,0,0)',\r\n    yaxis_title="млрд руб.",\r\n    shapes=dict_list\r\n)\r\nfig.update_xaxes(tickangle=-45, tickfont=dict(family='Open Sans, light', color='black', size=14))\r\nfig.update_yaxes(tickangle=0, tickfont=dict(family='Open Sans, light', color='black', size=14))\r\n\r\nfig.show()<\/code><\/pre>
And here it is:<\/p>\n