{"componentChunkName":"component---src-templates-blog-list-template-js","path":"/engineering/49","result":{"data":{"allMarkdownRemark":{"edges":[{"node":{"excerpt":"As my previous blog, I use the python web Crawler library to help crawl the static website. For the Scrapy, there can be customize download…","fields":{"slug":"/engineering/write-a-highly-efficient-python-web-crawler/"},"html":"

As my previous blog, I use the python web Crawler library to help crawl the static website. For the Scrapy, there can be customize download middle ware, which can deal with static content in the website like JavaScript.

\n

However, the Scrapy already helps us with much of the underlying implementation, for example, it uses it own dispatcher and it has pipeline for dealing the parsing word after download.  One drawback for using such library is hard to deal with some strange bugs occurring because they run the paralleled jobs.

\n

For this tutorial, I want to show the structure of a simple and efficient web crawler.

\n

First of all, we need a scheduler, who can paralleled the job. Because the most of the time is on the requesting.  I use the  gevent to schedule the jobs. Gevent uses the libevent as its underlying library, which combines the multithreading and event-based techniques to parallel the job.

\n

There is the sample code:

\n
import gevent\nfrom gevent import Greenlet\nfrom gevent import monkey\nfrom selenium import webdriver\nmonkey.patch_socket()\nclass WebCrawler:\n    def __init__(self,urls=[],num_worker = 1):\n        self.url_queue = Queue()\n        self.num_worker = num_worker\n    def worker(self,pid):\n        driver = self.initializeAnImegaDisabledDriver()  #initilize the webdirver\n#TODO catch the exception\n        while not self.url_queue.empty():\n            url = self.url_queue.get()\n            self.driver.get(url)\n            elem = self.driver.find_elements_by_xpath("//script | //iframe | //img") # get such element from webpage\n    def run(self):\n        jobs = [gevent.spawn(self.worker,i) for i in xrange(self.num_worker)]
\n

The next part is the headless browser part. I use the phantomjs with --webdriver=4444 --disk-cache=true --ignore-ssl-errors=true --load-images=false --max-disk-cache-size=100000. You can get the detailed option from their documents.

\n

Phantomjs uses selenium webdriver as front-end to handle the request. However phantomjs is using the webkit and QT as its underlying browser and controller. It has memory leak bugs therefore the phantomjs will consume ton of memory and it only can use one core of your CPU but you can deploy many instances of the phantomjs on different ports. I wrote a daemon process to monitor the memory and its situation but later I realize I can use Perl script to get the status of process and when it exceeds the limits like 1G memory and send kill signal to the process.

\n

To speed up the crawler, I choose to use static browser to verify the website first because the website is bad written, there might be deadlock occurring so just skip them.

\n","frontmatter":{"date":"July 14, 2015","updated_date":null,"description":null,"title":"Write a highly efficient python Web Crawler","tags":["Python","Coding"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/c068d4f41cc4f2a33e3bd2737ca105df/7fbdd/python-web-crawler.webp","srcSet":"/static/c068d4f41cc4f2a33e3bd2737ca105df/61e93/python-web-crawler.webp 200w,\n/static/c068d4f41cc4f2a33e3bd2737ca105df/1f5c5/python-web-crawler.webp 400w,\n/static/c068d4f41cc4f2a33e3bd2737ca105df/7fbdd/python-web-crawler.webp 610w","sizes":"(max-width: 610px) 100vw, 610px"}}},"author":{"id":"Mark Duan","github":null,"avatar":null}}}},{"node":{"excerpt":"The  memcached is one of the most popular open source on-memory key-value caching systems. I will briefly talk about the design of memory…","fields":{"slug":"/engineering/memcach-memory-management/"},"html":"

The  memcached is one of the most popular open source on-memory key-value caching systems. I will briefly talk about the design of memory management of memcached.

\n

Chunk and Slab

\n

\"memcached-1\"

\n
/* powers-of-N allocation structures */\ntypedef struct {\n    unsigned int size;      /* sizes of items */\n    unsigned int perslab;   /* how many items per slab */\n    void *slots;           /* list of item ptrs */\n    unsigned int sl_curr;   /* total free items in list */\n    unsigned int slabs;     /* how many slabs were allocated for this class */\n    void **slab_list;       /* array of slab pointers */\n    unsigned int list_size; /* size of prev array */\n    unsigned int killing;  /* index+1 of dying slab, or zero if none */\n    size_t requested; /* The number of requested bytes */\n} slabclass_t;
\n

This is the struct declaration of slabclass_t. Each slab class contains the same size of chunk, but different classes have different chunk sizes. The size is calculated by this algorithm:

\n
<!--while (++i < POWER_LARGEST && size
\n

The content value factor is defined when memcached memory is deployed with -f, which can change the size between slab classes. For this loop, the size is multiplied by a specified factor.

\n

\"memcached-2\"

\n

You can get this information by adding -vvv and you can use the command

\n
stats slabs under telnet connection ip port
\n

Every time when a new memory needs to be allocated. It will scan the slab class to find the most suitable class to store the chunk.

\n

Rebalance and reassign slab memory:

\n

From memcached’s wiki:

\n

Overview: Memcached 1.4.11. Fixes race conditions and crashes introduced in 1.4.10. Adds the ability to rebalance and reassign slab memory.

\n

Slab Reassignment

\n

Long running instances of memcached memory may run into an issue where all available memory has been assigned to a specific slab class (say items of roughly size 100 bytes). Later the application starts storing more of its data into a different slab class (items around 200 bytes). Memcached could not use the 100 byte chunks to satisfy the 200 byte requests, and thus you would be able to store very few 200 byte items.

\n

1.4.11 introduces the ability to reassign slab pages. This is a beta feature and the commands may change for the next few releases, so please keep this in mind. When the commands are finalized they will be noted in the release notes

\n

Slab Automove

\n

While slab reassign is a manual feature, there is also the start of an automatic memory reassignment algorithm.

\n

From the source code in slabs.c we can see, memcached uses two threads to monitor the slabs class, one is to do maintenance and another one is to do the re-balance the class.

\n

Memcached defines a global variable struct slab\\rebalance slab_rebal,_ which is used to store the start, end information of slab. s_clsid is the source slab id and d_clsid is the destination slab id. The detailed blog in Chinese memcached源码分析—–slab automove和slab rebalance could be helpful.

\n
struct slab_rebalance {\n    void *slab_start;\n    void *slab_end;\n    void *slab_pos;\n    int s_clsid;\n    int d_clsid;\n    int busy_items;\n    uint8_t done;\n};
\n

Memory Pool

\n

Memcached implements its own memory pool, which is used to avoid system memory allocation and memory fragmentation. That will make your memory efficient and easy to manage. Here is a demo implementation of memory pool. Basically it is a large pre-allocated chunk of memory.

\n
 mem_avail) {\n        return NULL;\n    }\n    /* mem_current pointer _must_ be aligned!!! */\n    if (size % CHUNK_ALIGN_BYTES) {\n        size += CHUNK_ALIGN_BYTES - (size % CHUNK_ALIGN_BYTES);\n    }\n    mem_current = ((char*)mem_current) + size;\n    if (size
\n","frontmatter":{"date":"July 07, 2015","updated_date":null,"description":null,"title":"Memcached Memory Management","tags":["Memory Management"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/283a834484b5b25cf8471e04a8e2ff1b/7fbdd/memcached-memory-management.webp","srcSet":"/static/283a834484b5b25cf8471e04a8e2ff1b/61e93/memcached-memory-management.webp 200w,\n/static/283a834484b5b25cf8471e04a8e2ff1b/1f5c5/memcached-memory-management.webp 400w,\n/static/283a834484b5b25cf8471e04a8e2ff1b/7fbdd/memcached-memory-management.webp 610w","sizes":"(max-width: 610px) 100vw, 610px"}}},"author":{"id":"Mark Duan","github":null,"avatar":null}}}},{"node":{"excerpt":"HTML 5 is the latest and greatest web technology, although it has some issues in some browsers which don’t have native support for the new…","fields":{"slug":"/engineering/html5-limitation-in-internet-explorer/"},"html":"

HTML 5 is the latest and greatest web technology, although it has some issues in some browsers which don’t have native support for the new HTML5 elements. If you want to support some of the older browsers which may still be in use such as IE8 and lower then you are going to have some trouble using HTML5.

\n

The problem with IE8 and lower versioned IE browsers is that they were created many years ago while HTML5 is new technology and HTML5 tags were newly invented so legacy browsers do not support them. Microsoft will not release any updates for these browsers because they have released new versions of the IE browser.

\n

As old IE browsers do not know about these tags then it can’t style them so you will find that in IE8 or lower your websites will be rendered without many styles.

\n

How to use HTML5 in IE

\n

If you want HTML5s new tags to be supported in IE then you need to add some javascript to create the elements:

\n

Example:

\n
  <script>\n    document.createElement('header');\n    document.createElement('article');\n    document.createElement('aside');\n    document.createElement('footer');\n  </script>
\n

This is not a very good way, you would need to remember all the tags that you have set and include them on every page of your website. If you add any new tags then you need to add these tags on every page of the website as well. Thankfully Remy Sharp provided a solution for this, Remy Sharp has created a Javascript file which he hosts with google code that will handle this for you.

\n

add the following at the top of your pages and it will include this code file on your website if the end user is using an IE browser lower than version 9.

\n
<!--[if lt IE 9]>\n<script src="https://html5shim.googlecode.com/svn/trunk/html5.js"></script>\n<![endif]-->
\n

IE also does not support some of the most useful new HTML5 methods that are supported by modern browsers:Window.postMessage()

\n

Window.postMessage

\n

allows for sending data messages between two windows/frames across domains. Normally, scripts on different pages are allowed to access each other if and only if the pages that executed them are at the same locations with the same protocol (usually both http) and matching host names. The HTML5 postMessage function provides a great way to communicate securely and with high performance and reliability even cross-domain. Unfortunately, the HTML5 postMessage function does not work in IE

\n

Let’s take a look at how Window.postMessage works:

\n

Syntax

\n

The below signature is used for postMessage:

\n

otherWindow.postMessage(message, targetOrigin);

\n

otherWindow\nA reference to another window; such a reference may be obtained, for example, using the contentWindow property of an iframe element, the object returned by window.open

\n

message\nThis parameter is required for a message to be sent to the other window. The message is serialized using the structured clone algorithm. This means you can pass a broad variety of data objects safely to the destination window without having to serialize them yourself.

\n

targetOrigin\nWhen the message is dispatched, the current location of the target document is checked. If it does not match the specified URI, then the message will not be dispatched. This parameter can be useful if you want to be sure of the location of the target document before dispatching the message and you can also specify a literal string “*”  (indicating no preference) or as a URI

\n

but always provide a specific targetOrigin, not *, if you know where the other window’s document should be located. If you do not set a specific location and use “*” then a malicious site can change the location of the window without your knowledge, and therefore it can intercept the data sent using the postMessage method.

\n

Example for postMessage

\n

Sender

\n

The first part of the process is setting up a “source”.  With the source, we will open a new window (or IFrame), send the new window message (in the our example, we’ll do so every 6 seconds, and create an event listener for any response we receive from the destination window.

\n
//create popup window\nvar domain = 'https://www.example.com';\nvar myPopup = window.open(domain + '/postmessage.html','myWindow');\n//periodical message sender\nsetInterval(function(){\n    var message = 'Hello!  The time is: ' + (new Date().getTime());\n    myPopup.postMessage(message,domain); //send the message and target URI\n},6000);\n \n//listen to back\n \nwindow.addEventListener('message',function(event) {\n    if(event.origin !== 'http://scriptandstyle.com') return;\n        console.log('received response:  ',event.data);\n},false);
\n

I have used window.addEventListener which doesn’t work with Internet Explorer so use window.attachEvent

\n

Receiver

\n

In the destination window we should validate the message origin and if it is not valid then we will not send the message to the sender, otherwise we send a response back to the sender:

\n
window.addEventListener('message',function(event) {\n    if(event.origin !== 'https://lrblogs.wpengine.com) return;\n        event.source.postMessage('message received:  ',event.origin);\n},false);
\n

sessionStorage

\n

sessionStorage stores data for one session only, this is used in a single transaction it stores the data only for one session and as you close the window the session would be lost and  the data is deleted when the browser is closed.

\n

HTML5 sessionStorage object are shared in different tabs in the same browser session if you change a sessionStorage attribute’s value in one tab, that change should be reflected within another tab but in IE8 this system will not work properly it does not share sessionStorage objects between frames on a page. This issue has since been fixed in IE11.

\n

Example:

\n
//save a value\nsessionStorage.setItem("website", "https://www.example.com");\n//retrieve item\nvar website= sessionStorage.getItem("website");\n//remove the key\nsessionStorage.removeItem(“website”);
\n

localStorage

\n

The localStorage object stores the data like a persistent cookie, with no expiration date. The data will not be deleted when the browser is closed, and will be available when a user returns to the browser.

\n

IE also supports localStorage from IE8 but it does not support localStorage in IE7 and previous versions.

\n

localStorage Vs Cookies

\n
    \n
  1. Cookies are small text files stored by browsers allowing for a max of 4KB while with localStorage we can store Mbs of localStorage data.
    2. \n
  2. Cookies are delivered with every request, which can slow down the delivery of your web pages.
    3. \n
\n

Example

\n
//save a value\nlocalStorage.setItem("Domain", "https://www.example.com");\n//retrieve item\nvar website= localStorage.getItem("website");\n//remove the key\nlocalStorage.removeItem(“website”);
\n","frontmatter":{"date":"June 30, 2015","updated_date":null,"description":null,"title":"HTML5 Limitation in Internet Explorer","tags":["Java","Maven","Eclipse"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/20127eac7a49d6d333dd2483a84b5c4c/7fbdd/radio-check-buttons-css.webp","srcSet":"/static/20127eac7a49d6d333dd2483a84b5c4c/61e93/radio-check-buttons-css.webp 200w,\n/static/20127eac7a49d6d333dd2483a84b5c4c/1f5c5/radio-check-buttons-css.webp 400w,\n/static/20127eac7a49d6d333dd2483a84b5c4c/7fbdd/radio-check-buttons-css.webp 610w","sizes":"(max-width: 610px) 100vw, 610px"}}},"author":{"id":"Team LoginRadius","github":"LoginRadius","avatar":null}}}},{"node":{"excerpt":"An API is an acronym for Application Programming Interface. API's include various tools, protocols, and routines. API's are used as a way…","fields":{"slug":"/engineering/what-is-an-api/"},"html":"

An API is an acronym for Application Programming Interface. API's include various tools, protocols, and routines. API's are used as a way for applications to communicate with various systems.

\n

But what does this mean? Well let me explain what they are with some examples. I am going to be using various API's for these examples.

\n

Let's Begin

\n

Let's start by looking at a favorite website of mine called The Yoda Speak Generator found here.

\n

This website allows users to enter text and get a return of that same text formatted the way Yoda might say it. This is done with an API.

\n

Yoda Speak: Fun stuff, api's are! Yes, hmmm.

\n

API’s contain what is called an endpoint to access this API. The endpoint provides the connection to communicate with the API and give it some text to generate into Yoda Speak. The API would then return the generated Yoda Speak to the location of the request.

\n

Another way to think of an API is to think of it like an egg carton. An egg carton holds the eggs, protects them from damage and allows someone who wants to make breakfast a way to open and close the carton to access the eggs inside. The eggs inside would be the data, and the hand reaching into the carton would be much like the communication using an endpoint from a server.

\n

Okay, Let's make breakfast.... Just Kidding!

\n

For those of you who are visual Let's look at some examples of the code. These snippets are in PHP because it is my favorite language. Below is a link to download a small zip file called apidemo.zip.

\n

Demo Link

\n

You can host this directory on your localhost or on a website if you choose.

\n

After you have extracted the zip and have a directory such as localhost/apidemo open a browser and enter http://localhost/apidemo/api.php?request=example&name=yourname where localhost is your domain.

\n

To enter this endpoint in an application that makes API calls try Postman. Here is a link for a chrome extension.

\n

Try using a GET and a POST method with parameters request=example for the example function in api.php and name=yourname to output a name

\n

You should now see the output \"Your name is yourname\" in your browser

\n

explore the files api.php, API.class.php and MyAPI.php to explore how this simple API is taking in the parameters and outputting a name

\n

Hopefully, I have explained API's so you have a basic understanding of what they are. There are many types of API's, one that you may see often is RESTful API's, these unlike this example must abide by constraints that have been defined by the REST architecture.

\n","frontmatter":{"date":"June 23, 2015","updated_date":null,"description":null,"title":"What is an API","tags":["API","Postman"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/a01dd1d7515afffa79ea80811d70b895/403a4/whats-an-api-300x300.webp","srcSet":"/static/a01dd1d7515afffa79ea80811d70b895/61e93/whats-an-api-300x300.webp 200w,\n/static/a01dd1d7515afffa79ea80811d70b895/403a4/whats-an-api-300x300.webp 300w","sizes":"(max-width: 300px) 100vw, 300px"}}},"author":{"id":"Zoie Carnegie","github":null,"avatar":null}}}},{"node":{"excerpt":"Let's face it, at some point you look at a radio or a checkbox button and you're like... this looks like something that starts with an \"s…","fields":{"slug":"/engineering/styling-radio-and-check-buttons-with-css/"},"html":"

Let's face it, at some point you look at a radio or a checkbox button and you're like... this looks like something that starts with an \"s\" and ends with a \"hit\". I'm here to stop you from saying that word and go on with your life without worrying about those ugly things.

\n

We're just going to use code here. No images, just pure scalable user interface using code. Cool, eh? Let's get started.

\n

Let's create our code structure first.

\n
<!-- Radio --> \n<label>\n<input type="radio">\n<span>I'm a radio button</span>\n</label>\n<!-- Checkbox -->\n<label>\n<input type="checkbox">\n<span>I'm a checkbox</span>\n</label>
\n

If it looks ugly, don't be alarmed. That's normal for an untamed code from the wild. All you gotta do now is to tame it with our very best tool, CSS.

\n

First thing you have to do is to hide the element that generates the hideous button, which is... DUN DUN DUN... the input tag. Don't worry, we can still trigger it even though it's hidden. How? by nesting it inside the label tag.

\n

Now that the default toggles are gone, we need to create our fake —but amazing— toggles.

\n

Radio Button

\n

We can't really customize this toggle that much because it needs other toggles to work properly. Let's just prettify it a bit.

\n
label input[type="radio"] {\n    display: none;\n}\nlabel input[type="radio"] ~ span {\n    position: relative;\n    display: inline-block;\n    padding: 3px 0 3px 25px;\n}\nlabel input[type="radio"] ~ span:before {\n    content: "";\n    position: absolute;\n    display: block;\n    width: 18px;\n    height: 18px;\n    background: #fff;\n    margin-right: 5px;\n    border: 1px solid #ccc;\n    border-radius: 50%;\n    left: 0;\n    top: 0;\n    box-sizing: border-box;\n    transition: all 300ms ease-in-out;\n}\nlabel input[type="radio"]:checked ~ span:before {\n    border: 5px solid #29d;\n}
\n

\"checked-radio\"

\n

\"unchecked-radio\"

\n

By now it should like this.

\n

Cleaner toggles!

\n

Checkbox

\n

This one is fun to customize because it has an on and off feature or a switch. Let's start with a simple toggle. We're going to transform this hideous toggle (screenshot) to this beautiful creature (screenshot).

\n
label.spin input[type="checkbox"] {\n    display: none; /*hides ugly toggle*/\n}\nlabel.spin input[type="checkbox"] ~ span {\n    position: relative;\n    display: inline-block;\n    padding: 3px 0 3px 25px; /*adds spacing on the left*/\n}\n/*create our new toggle*/\nlabel.spin input[type="checkbox"] ~ span:before {\n    content: "\\2713"; /*add a new check mark*/\n    text-align: center;\n    font-size: 13px;\n    position: absolute;\n    display: block;\n    width: 18px;\n    height: 18px;\n    background: #fff;\n    color: #fff;\n    margin-right: 5px;\n    border: 1px solid #ccc;\n    border-radius: 50%;\n    left: 0;\n    top: 0;\n    box-sizing: border-box;\n    transition: all 500ms ease-in-out;\n}\n/*if checked do this*/\nlabel.spin input[type="checkbox"]:checked ~ span:before {\n    transform: rotatez(360deg);\n    background: #29d;\n    border-color: #29d;\n}
\n

Oh yes! Beautiful toggle.

\n

\"checked-check\"

\n

\"unchecked-checkbox\"

\n

Bonus!

\n

Have you ever seen those toggles where it slides to the left and right when you click it? Did you know you can do that too using CSS? Let's try it.

\n

First, we're going to design the default look of an unchecked checkbox. By now you know that we need to hide the default toggle.

\n
label input[type="checkbox"] { display: none; }
\n

And then add a space on the left of the span using padding. While you're there, add a position relative to because we need to contain the rest of the elements inside the span.

\n
label input[type="checkbox"] ~ span { position: relative; display: inline-block; padding: 3px 0 3px 35px; }
\n

Now, let's create our toggles using CSS pseudo element :before and :after. Why? Because generated CSS is easier to handle.

\n
label input[type="checkbox"] ~ span:before, label.slide input[type="checkbox"] ~ span:after {\n    content: "";\n    position: absolute;\n    display: block;\n    width: 18px;\n    height: 18px;\n    background: #fff;\n    margin-right: 5px;\n    border: 1px solid #ccc;\n    left: 0;\n    top: 0;\n    box-sizing: border-box;\n    transition: all 300ms ease-in-out;\n}
\n

This will generate a box looking element. Since it's targeting both pseudo elements, they will look identical and on top of each other.

\n

It's time to start prettifying those pseudo elements.

\n

For :after, since this is the background where the :before toggle slides through. We need to modify the width to look bigger. Add an inset shadow to make it look like it's pushed through and a red background for an \"off\" effect.

\n
label input[type="checkbox"] ~ span:before {\n    width: 30px;\n    border-radius: 20px;\n    box-shadow: inset 0 2px 5px -1px rgba(0, 0, 0, 0.4), inset 0 -2px 0 -1px rgba(255, 255, 255, 0.2);\n    background: #F22613;\n}
\n

For :before, we just need to make it look like a circle and lift it off a bit using box-shadow.

\n
label.slide input[type="checkbox"] ~ span:after {\n    box-shadow: 0 3px 4px -2px rgba(0, 0, 0, 0.5);\n    border-radius: 50%;\n}
\n

The \"on\" part just needs a little bit of pushing to the side and changing the background to blue or green.

\n
label.slide input[type="checkbox"]:checked ~ span:before {\n    background: #29d;\n}\nlabel.slide input[type="checkbox"]:checked ~ span:after {\n    left: 13px;\n}
\n

The CSS is a bit longer here because there's a lot of elements that need to change, but take a chill pill because CSS won't slow your page down.

\n

And that's it. Your pretty checkbox or radio button toggles.

\n","frontmatter":{"date":"June 16, 2015","updated_date":null,"description":null,"title":"Styling Radio and Check buttons with CSS","tags":["CSS"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/20127eac7a49d6d333dd2483a84b5c4c/7fbdd/radio-check-buttons-css-1.webp","srcSet":"/static/20127eac7a49d6d333dd2483a84b5c4c/61e93/radio-check-buttons-css-1.webp 200w,\n/static/20127eac7a49d6d333dd2483a84b5c4c/1f5c5/radio-check-buttons-css-1.webp 400w,\n/static/20127eac7a49d6d333dd2483a84b5c4c/7fbdd/radio-check-buttons-css-1.webp 610w","sizes":"(max-width: 610px) 100vw, 610px"}}},"author":{"id":"Team LoginRadius","github":"LoginRadius","avatar":null}}}},{"node":{"excerpt":"Social sharing button offers an easy way to distribute your content to a wider audience. The major social providers allow options to…","fields":{"slug":"/engineering/configuring-social-sharing-buttons/"},"html":"

Social sharing button offers an easy way to distribute your content to a wider audience. The major social providers allow options to configure the content that will be shared programmatically. In this guide we explore the options that you can use in order to configure the shared content of your social shares including: Message, URL, Image, and Image Caption/Description.

\n

Facebook

\n

Facebook utilizes Open Graph Tags which are meta tags that can be included on your page and are scrapped by Facebook when determining the content that should be shared with a specific button. These tags can be included in the head section of your page. All meta tags should be formatted with a property value of \"og:\", some useful tag names are:

\n\n

Example of og meta tags that can be included in your html head tag:

\n
<!--\n\n\n\n\n\n-->
\n

If you are modifying these tags it can take some time for the changes to translate over to Facebook. You can refer to this blog on troubleshooting, testing, and force updating your configured sharing systems.

\n

Google

\n

Google offers multiple ways that you can set the shared content which it intelligently determines the most relevant details to be displayed in the share. This works to varying degrees with the following options available and taking precedence respectively:

\n\n

Below is an example of basic structured data markup:

\n
<!--\n Your Site Title \n\n\n    A description of your site.\n\n-->
\n

Below is an example of using basic meta tags to define the shared content:

\n
<!--\n\n-->
\n

Twitter

\n

The best way to customize your Twitter shares is to modify the URL query string parameters that gets triggered when the share icon is clicked. Another way is to customize the <a> tag by including data-attribute tags as detailed on Twitters Tweet button creation instructions . If you do not have control over the links or triggers that are being used to activate the Tweet popup you can also set Twitter Card meta tags which handle the Twitter customizations.

\n
<!--\n\n\n\n\n-->
\n

Various providers will utilize some of the above methods with open graph tags being one of the most common methods for controlling the shared content. Other systems will customize this content based off of parameters that are passed into the script or URL that is used to trigger the share interface. Getting these to be correctly configured can be a bit of a trial and error process, you can use the resources detailed in this blog post to help test and verify that you have correctly configured your social sharing content.

\n","frontmatter":{"date":"June 09, 2015","updated_date":null,"description":null,"title":"Configuring Your Social Sharing Buttons","tags":["SocialSharing"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/24799a6bff48ea3ac4e96368c579f88b/403a4/configuring-social-share-buttons-300x300.webp","srcSet":"/static/24799a6bff48ea3ac4e96368c579f88b/61e93/configuring-social-share-buttons-300x300.webp 200w,\n/static/24799a6bff48ea3ac4e96368c579f88b/403a4/configuring-social-share-buttons-300x300.webp 300w","sizes":"(max-width: 300px) 100vw, 300px"}}},"author":{"id":"Karl Wittig","github":null,"avatar":null}}}}]},"markdownRemark":{"excerpt":"Introduction Ever wondered how apps like Spotify, Netflix, or Slack manage seamless login experiences across devices? Many of them use JWT…","fields":{"slug":"/engineering/how-to-integrate-jwt/"},"html":"

Introduction

\n

Ever wondered how apps like Spotify, Netflix, or Slack manage seamless login experiences across devices? Many of them use JWT, or JSON Web Tokens, a compact, stateless method for securely transmitting user identity and session data across services.

\n

With JWT token authentication, identity information is embedded in a signed token, allowing you to maintain user sessions without server-side storage. This approach is highly scalable and ideal for modern architectures like SPAs, mobile apps, and microservices.

\n

In this blog, we’ll walk you through what is JWT, why use it, and how to implement JWT authentication using LoginRadius.

\n

You’ll learn what JWT is, why it’s effective, and how it works in real-world applications. We'll cover both integration methods (IDX and Direct API), generating your signing key, managing sessions, storing the JWT token securely, and applying best practices throughout.

\n

Whether you're a developer, product manager, or IAM architect, this guide offers a complete foundation for implementing JWT token authentication into your application stack.

\n

What is JWT?

\n

JSON Web Token (JWT) is an open standard (RFC 7519) used to transmit information securely between parties as a JSON object. It’s compact, self-contained, and digitally signed, making it a reliable format for authentication and authorization across modern applications.

\n

A JWT consists of three parts:

\n
    \n
  1. Header – Contains metadata like the type of token and signing algorithm (e.g., HS256).
    2. \n
  2. Payload – Stores the actual data or “claims,” such as user ID, roles, and token expiry.
    3. \n
  3. Signature – A cryptographic hash that ensures the token hasn’t been tampered with.
    4. \n
\n

Example of a token structure:

\n

<base64Header>.<base64Payload>.<signature>

\n

Why Use JWT?

\n\n

How JWT Works?

\n

\"Flowchart

\n
    \n
  1. A user logs in with credentials.
    2. \n
  2. Your app (or identity provider like LoginRadius) issues a signed JWT.
    3. \n
  3. The client stores the token and sends it with each request (usually in the Authorization header).
    4. \n
  4. The server validates the token’s signature and claims.
    5. \n
  5. If valid, access is granted — without any session stored on the backend.
    6. \n
\n

JWT simplifies identity verification, especially when you're building apps that talk to APIs or need to scale without centralized session storage.

\n

JWT Authentication with LoginRadius: Overview

\n

LoginRadius provides robust support for JWT (JSON Web Token) authentication, which allows for flexible and secure access control across different digital platforms. Whether you're building a fully custom identity flow or using a pre-built interface, the platform supports various integration approaches depending on your architecture.

\n

If you're looking to understand how to implement JWT token authentication effectively, LoginRadius offers two primary implementation models that cater to different levels of customization and control:

\n

1. IDX Implementation – JWT through a Hosted Login Page

\n

The IDX-hosted login approach enables secure, standards-compliant, JWT-based authentication without requiring you to build a custom login interface. This is a strategic option for fast, compliant, and user-friendly deployments.

\n\n

Configuration Steps:

\n
    \n
  1. Enable JWT Login
    2. \n
  2. Go to authentication configuration settings and enable JWT Login in the Admin Console.
    3. \n
\n

\"Screenshot

\n
    \n
  1. Specify your signing algorithm and expiry policy, and define your JWT Secret Key.
    2. \n
  2. Input a secure JWT signing key.
    3. \n
  3. Specify token expiry duration (e.g., 15–60 minutes)
    4. \n
  4. Select the desired algorithm —HS256 for symmetric signing (same key signs and verifies)
    5. \n
  5. RS256 for asymmetric signing, where LoginRadius securely stores the private key used to sign the JWT.
    6. \n
  6. Your app or backend service uses the public key to validate the token signature.
    7. \n
  7. LoginRadius provides a JWKS (JSON Web Key Set) endpoint to dynamically fetch and rotate public keys, ensuring trust without key exposure.
    8. \n
  8. Update IDX Template for Callback
    9. \n
  9. Modify your IDX login page template to retrieve the JWT post-login. You can access the token via redirect URL parameters or secure JavaScript callbacks.
    10. \n
\n

Example Response:

\n

{

\n

\"access_token\": \"eyJhbGciOiJIUzI1NiIsInR...\",

\n

\"expires_in\": 1800

\n

}

\n

This integration approach works best for all teams that want effective identity workflows without the complexity of building proprietary login screens, something that is crucial for customer portals, onboarding of mobile applications, and even managing access for business partners.

\n

2. Direct API Implementation – Self Managed Login

\n

If you’re building a custom login UI or working in a headless environment, LoginRadius lets you generate and handle JWTs directly through its Authentication APIs. Here’s how you can programmatically perform token authentication using the classic method:

\n\n

Configuration Steps:

\n

Step 1: Authenticate via API:

\n\n

Include the user’s credentials (email + password) in the request body.

\n

Step 2: Get JWT in Response

\n\n

{

\n

\"access_token\": \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...\",

\n

\"expires_in\": 3600

\n

}

\n

Step 3: JWT Decoding and Validation

\n\n

Step 4: Set Custom Claims (Optional)

\n

With LoginRadius, it is possible to customize the payload to include user roles and/or any additional metadata. You can set custom JWT claims on the Admin Console.

\n

With this method, you have complete customization over login flows while using LoginRadius to issue signed JWTs for user session management.

\n

NOTE- With either method, LoginRadius ensures that JWTs are securely signed, optionally short-lived, and compatible with standard token validation libraries, making integration seamless for everyone.

\n

To get started with JWT implementation, you can read our complete developer documentation.

\n

Hosted Login vs Direct API

\n

\"Illustration

\n

What is Session Management and How It Works with JWT

\n

Session management is how your app keeps track of a user after they log in so they don’t have to prove who they are with every request.

\n

In traditional apps, sessions are stored on the server using session IDs. Every time a request comes in, the server checks that session ID to verify the user.

\n

In modern apps, especially SPAs and APIs, JWTs are used to manage sessions without needing server-side storage; this is called stateless session management. The token itself carries the user’s identity, roles, and expiration details. As long as the token is valid, the user stays logged in.

\n

Good session management ensures:

\n\n

How LoginRadius Handles Session Management with JWT:

\n
    \n
  1. \n

    User Logs In

    \n\n
    2. \n
  2. \n

    Client Stores the Token

    \n\n
    3. \n
  3. \n

    Access Token Expiry

    \n\n
    4. \n
  4. \n

    Token Renewal

    \n\n
    5. \n
  5. Logout and Token Revocation Strategy
    6. \n
\n

When the user logs out, both the access token and refresh token should be cleared from client storage.

\n\n

Combining these strategies gives you greater control over token misuse and enables a robust, enterprise-grade logout flow.

\n

\"illustration

\n

How to Store JWT Tokens?

\n

When you implement JWT-based authentication, the client (browser or mobile app) needs a way to store the access token and, optionally, the refresh token after they are issued by the authentication server. This stored token is then attached to every subsequent request to prove the user's identity.

\n

Choosing where to store the JWT is a crucial security decision. The most common storage options are:

\n\n

Each option has trade-offs between security, accessibility, and persistence, and the right choice depends on your application's architecture and threat model.

\n

Recommended Storage Strategy

\n\n

Best Practices for Storing JWTs

\n
    \n
  1. Never store sensitive tokens (like refresh tokens) in localStorage or sessionStorage.
    2. \n
  2. Use Secure and HttpOnly flags with cookies to prevent JavaScript access and ensure transmission only over HTTPS.
    3. \n
  3. Set the SameSite=Strict or Lax attribute on cookies to protect against CSRF.
    4. \n
  4. Use short-lived access tokens and rotate refresh tokens regularly.
    5. \n
  5. Implement CSP (Content Security Policy) to reduce XSS risk.
    6. \n
  6. Avoid storing any tokens in frontend code (e.g., hardcoded in JS files).
    7. \n
\n

Conclusion

\n

JWT authentication with LoginRadius offers a modern, stateless approach to managing sessions across distributed systems. The IDX integration is ideal for rapid deployment, while the Direct API model is best for organizations needing deep customization and integration flexibility.

\n

With robust token signing, refresh capabilities, and centralized control, LoginRadius provides a future-ready foundation for secure, scalable identity architecture. Contact us to know more about JWT authentication and implementation guide.

\n

FAQs

\n

1. What is JWT authentication used for?

\n

A: JWT authentication securely verifies user identities, enabling stateless session management across web, mobile apps, and microservices without server-side session storage.

\n

2. How does LoginRadius simplify JWT integration?

\n

A: LoginRadius simplifies JWT integration by offering hosted IDX login pages and direct API-based authentication methods, enabling rapid deployment and deep customization.

\n

3. Is JWT authentication secure?

\n

A: Yes, JWT authentication is secure when implemented with best practices like short-lived tokens, secure storage methods, signature validation, and refresh token rotation.

\n

4. Can JWT tokens be revoked with LoginRadius?

\n

A: Yes, LoginRadius allows revocation of JWT refresh tokens explicitly, and supports strategies like short-lived tokens and key rotation to manage token lifecycles securely.

\n","frontmatter":{"date":"April 15, 2025","updated_date":null,"description":"Discover JWT (JSON Web Token) authentication, its advantages, and how to integrate it seamlessly using LoginRadius' hosted IDX and Direct API methods for secure, scalable identity management.","title":"JWT Authentication with LoginRadius: Quick Integration Guide","tags":["JWT","JSON Web Token","Authentication","Authorization"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":0.7782101167315175,"src":"/static/4cedb7829f98208cbc6d5a9aea4e983d/58556/how-to-integrate-jwt.webp","srcSet":"/static/4cedb7829f98208cbc6d5a9aea4e983d/61e93/how-to-integrate-jwt.webp 200w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/1f5c5/how-to-integrate-jwt.webp 400w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/58556/how-to-integrate-jwt.webp 800w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/1cc9f/how-to-integrate-jwt.webp 896w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Kundan Singh","github":null,"avatar":null}}}},"pageContext":{"limit":6,"skip":288,"currentPage":49,"type":"//engineering//","numPages":53,"pinned":"5c425581-f474-5ae9-abe7-cf5342db2aaa"}},"staticQueryHashes":["1171199041","1384082988","2100481360","23180105","528864852"]}