{"componentChunkName":"component---src-templates-blog-list-template-js","path":"/engineering/30","result":{"data":{"allMarkdownRemark":{"edges":[{"node":{"excerpt":"In this article, we will learn about state management in Angular with NGXS. Here we will understand the basic concept of NGXS and create a…","fields":{"slug":"/engineering/angular-state-management-with-ngxs/"},"html":"

In this article, we will learn about state management in Angular with NGXS. Here we will understand the basic concept of NGXS and create a simple TO-DO App.

\n

What is State

\n

In the bigger apps, managing the data states is very complicated. In angular, each component has its own state, to share the data/state between the components we normally use @Input and @Output decorators, but when the application goes bigger, its challenging to maintain the data consistency. So to solve this problem, redux was introduced. It provides a central store that holds all states of your application. Each component can access the stored state without sending it from one component to another.

\n

\"Image\"

\n

What is NGXS

\n

NGXS is a state management pattern + library. It provides as a single source of truth for your application's state, providing simple rules for predictable state mutations.

\n

NGXS is modeled after the CQRS pattern popularly implemented in libraries like Redux and NgRx but reduces boilerplate by using modern TypeScript features such as classes and decorators.

\n

How NGXS works:

\n

NGXS is very simple to use as compared to other state management patterns like redux and Akita. NGXS takes full advantage of angular and typescript over the redux pattern.\nThere are majors 4 concepts to NGXS

\n

1. Store:

\n

It is a global state container and manages the states of the application.We can dispatch the actions to perform certain operations.

\n
    this.store.dispatch(new TodoActions.AddTodo(form));
\n

2. Actions

\n

An action is a type of command which should be called when something happens or you want to trigger at any event like adding a new todo, listing todos etc.

\n
export class AddTodo {\n    static readonly type = '[Todo] Add';\n    constructor(public payload: ITodo) { }\n}
\n

3. State

\n

States are classes along with decorators to describe metadata and action mappings.

\n
import { Injectable } from '@angular/core';\nimport { State } from '@ngxs/store';\n\n@State<ITodoStateModel[]>({\n    name: 'todoList',\n    defaults: {\n        todoList: [],\n    },\n})\n@Injectable()\nexport class TodoState {}
\n

4. Select

\n

Selects are functions that slice a specific portion of the state from the global state container i.e. to get the data from the specific global state whenever you want to use.

\n
  @Select(TodoState) todoList$: Observable<ITodo>;
\n

Let's build a To-Do App:

\n

1. Get started

\n

Install the npm install @ngxs/store --save then import the below code in app.module.ts

\n
//File name app.module.ts\nimport { NgxsModule } from '@ngxs/store';\n@NgModule({\n  imports: [\n    NgxsModule.forRoot([ToDoState], { // here login state \n      developmentMode: !environment.production\n    })\n  ]\n})\nexport class AppModule {}
\n

2. Create your store

\n

Here we have created a component that dispatches actions to create a to-do and for other operations. Apart from that, we are using a selector TodoState, from which we are listening for the updated to-do list.

\n

Put this code in the app.component.ts

\n
// File name app.component.ts\nimport { Component } from '@angular/core';\nimport { Store, Select } from '@ngxs/store';\nimport { TodoActions } from './state/todo-actions';\nimport { FormGroup, FormControl, Validators } from '@angular/forms';\nimport { TodoState, ITodo } from './state/todo-state';\nimport { Observable } from 'rxjs';\n@Component({\n  selector: 'app-root',\n  templateUrl: './app.component.html',\n  styleUrls: ['./app.component.scss']\n})\nexport class AppComponent {\n  title = 'ngxs-todo-app';\n\n  @Select(TodoState) todoList$: Observable<ITodo>;\n\n  addForm = new FormGroup({\n    title: new FormControl('', [Validators.required])\n  });\n  constructor(private store: Store){}\n  onSubmit(form: any){\n    this.store.dispatch(new TodoActions.AddTodo(form));\n  }\n  markDone(id: string, is_done: boolean){\n    this.store.dispatch(new TodoActions.markDone(id, is_done));\n  }\n}
\n

3. Create your actions

\n

Create a folder state and put the todo-actions.ts file and add below code

\n
// File name todo-actions.ts\nexport class AddTodo {\n    static readonly type = '[Todo] Add';\n    constructor(public payload: any) { }\n}\n\nexport class EditTodo {\n    static readonly type = '[Todo] Edit';\n    constructor(public payload: any) { }\n}\n\nexport class FetchAllTodos {\n    static readonly type = '[Todo] Fetch All';\n}\n\nexport class DeleteTodo {\n    static readonly type = '[Todo] Delete';\n    constructor(public id: number) { }\n}
\n

4. Create your State:

\n

Here we have created the state of the todo, it contains the global state of the todo list.\nCreate a folder state and put the todo-state.ts file and add below code

\n
// File name todo-state.ts\nimport { Injectable } from '@angular/core';\nimport { State, NgxsOnInit, Action, StateContext } from '@ngxs/store';\nimport { TodoActions } from './todo-actions';\nimport { patch, updateItem } from '@ngxs/store/operators';\n\nexport interface ITodo {\n    id: string;\n    title: string;\n    is_done: boolean;\n}\nexport interface ITodoStateModel {\n    todoList: ITodo[];\n}\n@State<ITodoStateModel>({\n    name: 'todoList',\n    defaults: {\n        todoList: [],\n    },\n})\n@Injectable()\nexport class TodoState implements NgxsOnInit {\n    ngxsOnInit(ctx) {\n        ctx.dispatch(new TodoActions.FetchAllTodos());\n    }\n    @Action(TodoActions.markDone)\n    markDone(\n      ctx: StateContext<ITodoStateModel>, \n      { payload, is_done }: TodoActions.markDone\n    ) {\n        ctx.setState(\n            patch({\n                todoList: updateItem(\n                  (item: ITodo) => item.id === payload, \n                  patch({ is_done: !is_done })\n                )\n            })\n        );\n    }\n\n    @Action(TodoActions.AddTodo)\n    add(\n        ctx: StateContext<ITodoStateModel>,\n        { payload }: TodoActions.AddTodo,\n    ) {\n        const state = ctx.getState();\n        ctx.setState({\n            ...state,\n            todoList: [\n                ...state.todoList,\n                {\n                    ...payload,\n                    id: Math.random().toString(36).substring(7),\n                    is_done: false\n                }\n            ],\n        }\n        );\n    }\n\n}
\n

5. Create your html view

\n

Here we have created a form that we use to create todo and listed all todos.\nput this code in your app.component.html

\n
<!-- File name  app.component.html -->\n<form [formGroup]="addForm" (ngSubmit)="onSubmit(addForm.value)">\n  <input type="text" formControlName="title" class="form-control todo-list-input" placeholder="What do you need to do today?">\n  <button class="add btn btn-primary font-weight-bold todo-list-add-btn">Add</button> \n</form>\n<ul class="d-flex flex-column-reverse todo-list">\n    <li *ngFor="let todo of (todoList$ | async) ?. todoList" [ngClass]="{'completed': todo.is_done}">\n      <div class="form-check"> \n        <label class="form-check-label"> \n          <input (click)="markDone(todo.id, todo.is_done)" class="checkbox" type="checkbox" [checked]="todo.is_done">{{todo.title}} <i class="input-helper"></i></label> \n      </div>\n    </li>\n  </ul>
\n

After lots of code, it's time to see the results. So here is the UI you will get. To get the complete code please go to Github Repo.\n\"Image\"

\n

Conclusion

\n

We have gone through the basic concepts of NGXS and built a simple to-do app. There are a lot of advanced features and plugins by NGXS, but beyond the scope, we could not add, but hopefully, we will be exploring in future blogs.

\n","frontmatter":{"date":"September 08, 2020","updated_date":null,"description":"NGXS is a state management pattern + library for Angular. It acts as a single source of truth for your application's state, providing simple rules for predictable state mutations.","title":"Angular State Management With NGXS","tags":["Angular","NGXS","Redux"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.2903225806451613,"src":"/static/a6cc04de422582309bcb1c936a4244d6/58556/title-image.webp","srcSet":"/static/a6cc04de422582309bcb1c936a4244d6/61e93/title-image.webp 200w,\n/static/a6cc04de422582309bcb1c936a4244d6/1f5c5/title-image.webp 400w,\n/static/a6cc04de422582309bcb1c936a4244d6/58556/title-image.webp 800w,\n/static/a6cc04de422582309bcb1c936a4244d6/4ff9f/title-image.webp 875w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Akshaya Sharma","github":"akshay-js","avatar":null}}}},{"node":{"excerpt":"Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved…","fields":{"slug":"/engineering/what-is-the-new-go-1-15/"},"html":"

Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.

\n

As Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.

\n

Download the latest Go Version from Golang.

\n

Operating system and platform compatibility

\n

macOS

\n

Go 1.14 version announced some changes for the macOS. According to announcement

\n
\n

Go 1.14 is the last release that will run on macOS 10.11 El Capitan. Go 1.15 will require macOS 10.12 Sierra or later.

\n
\n

The lastest macOS 10.15 (Catalina) is no longer supporting 32-bit binaries. Go 1.14 will likely to be the last Go release to support 32-bit binaries on iOS, iPadOS, watchOS, and tvOS (the darwin/arm port). Go continues to support the 64-bit darwin/amd64 port.

\n

Windows

\n

Go now generates Windows ASLR executables when -buildmode=pie cmd/link flag is provided. Go command uses -buildmode=pie by default on Windows.

\n

The -race and -msan flags now always enable -d=checkptr, which checks uses of unsafe. Pointer. This was previously the case on all OSes except Windows.

\n

Android

\n

Mobile applications were crashing in some devices due to some invalid lld linker versions so now avoiding the crashing the latest Go 1.15 version explicitly selects the lld linker available in recent versions of the NDK. This explicitly lld linker avoids crashes on some devices. it becomes the default NDK linker in a future NDK version.

\n

Smaller binaries & Performance improvements

\n

Always new Go 1.15 brings many minor/major performance improvements. The first major improvement has reduced the executable's binary size by almost 5% - 10% as compared to Go 1.14. As Brad Fitzpatrick tweet, test program down from 8.2MB in Go 1.14 to 3.9MB in 1.15.

\n

\"image

\n

\"image

\n

Go 1.15 has improved the code build process now unused code is eliminating in the new linker, along with several targeted improvements, such as Clements's CL 230544, which reduces the number of stack and register maps included in the executable.

\n

Standard library additions

\n

Always in every new version release Go come with various minor changes and updates to the library. I m including some useful and important changes

\n

tzdata Package

\n

A new embedded tzdata package was added that permits embedding the timezone database into a program. Importing this package (as import _ \"time/tzdata\") which allows the program to find timezone information without the timezone database on the local system. We can also embed the timezone database by building with -tags timet/zdata. This approach increases the size of the program by about 800 KB. This might be useful and can test some code with the virtualized environments like Go playground.

\n

X.509 CommonName deprecation

\n

Go older versions were using CommonName field on X.509 certificates as a hostname if there is no Subject\nAlternative Names, would be disabled by default. If Still want to use this legacy behavior then we need to add x509ignoreCN=0 in the GODEBUG environment variable.

\n

net/url Package

\n

The net/url package adds a new URL. The redacted () method that returns the URL as a string. This is proposed in the Issue 34855. It is a very useful improvement for audit logging and security. It's a simple derivation from the URL.String() that masks the password if exists from the string being passed. It does not modify at all to the URL itself but a copy of it.

\n

Summary

\n

Go 1.15 has released with various improvements, bug fixes. We can get all improvements, closed proposals and features details for the Go GitHub repository.

\n

Referance

\n\n","frontmatter":{"date":"September 07, 2020","updated_date":null,"description":null,"title":"What's new in the go 1.15","tags":["Go","Golang","Go 1.15"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7699115044247788,"src":"/static/a12ace4cf8f8db7e673481e22dc14c33/58556/index.webp","srcSet":"/static/a12ace4cf8f8db7e673481e22dc14c33/61e93/index.webp 200w,\n/static/a12ace4cf8f8db7e673481e22dc14c33/1f5c5/index.webp 400w,\n/static/a12ace4cf8f8db7e673481e22dc14c33/58556/index.webp 800w,\n/static/a12ace4cf8f8db7e673481e22dc14c33/99238/index.webp 1200w,\n/static/a12ace4cf8f8db7e673481e22dc14c33/7c22d/index.webp 1600w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null}}}},{"node":{"excerpt":"Programming is an excellent exercise for your brain as programmers deal with never-ending situations like bugs, compiler, issues, deadlines…","fields":{"slug":"/engineering/lets-take-meme-break/"},"html":"

Programming is an excellent exercise for your brain as programmers deal with never-ending situations like bugs, compiler, issues, deadlines, languages, etc. During all these times, other than the phenomenal codes, we produce MEMEs and a hell lot of MEMEs

\n

MEMEs can light the environment for every programmer in the world. On that note, there is a famous saying-

\n

“If debugging is the process of removing software bugs, then programming must be the process of putting them in – Edsger Dijkstra”

\n

Here is our developer picked list of 21 complied MEME for you:

\n

1. Always ready for new!!! - Kunal

\n

\n

2. Nothing Changes With Time - Tanvi Jain

\n

\n

3. Heaviest Object In The Universe - Nitesh Jain

\n

\n

4. Perception!! - Vikram Jain

\n

\n

5. Maths Always A hurdle !! - Mohammed Modi

\n

\n

6. I had a problem so I thought to use Java - Indrasen Kumar

\n

\n

7. Avoid Handshakes. - Piyush Kumar

\n

\n

8. I’m Really Unsure About My Own Creation - Kheenvraj Lomror

\n

\n

9. That Missing Fact - Priyadarshan Mohanty

\n

\n

10. Every IT Administrator. - Narendra Pareek

\n

\n

11. Imma Genius! - Abhimanyu Singh Rathore

\n

\n

12. My Logical Thinking as Programmer​​​. - Ankit Singh

\n

\n

13. HTML is a Programming Language - Hitesh Kumawat

\n

\n

14. Programmers while Sleeping​. - Versha Gupta

\n

\n

15. Programmer Swag!! - Gaurav Bewal

\n

\n

16. QA Life!! - Rakesh Pareek

\n

\n

17. Infinite... - Hridayesh Sharma

\n

\n

18. Python Lover!! - Aman Agrawal

\n

\n

19. What's your address? - Vijay Singh Shekhawat

\n

\n

20. Gamer and Programmer - Chinmaya Pati

\n

\n

If you have better ones, reply in the comments below and we’ll include them.

\n","frontmatter":{"date":"September 04, 2020","updated_date":null,"description":null,"title":"Let’s Take A MEME Break!!!","tags":["MEME","Programming","Humor"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7241379310344827,"src":"/static/70f72136eb4f9f794700c1b2afd974c8/25338/cover.webp","srcSet":"/static/70f72136eb4f9f794700c1b2afd974c8/61e93/cover.webp 200w,\n/static/70f72136eb4f9f794700c1b2afd974c8/1f5c5/cover.webp 400w,\n/static/70f72136eb4f9f794700c1b2afd974c8/25338/cover.webp 550w","sizes":"(max-width: 550px) 100vw, 550px"}}},"author":{"id":"Ashish Sharma","github":"ashish8947","avatar":null}}}},{"node":{"excerpt":"PKCE is an OAuth 2.0 security extension for public clients on mobile devices intended to avoid a malicious programme creeping into the same…","fields":{"slug":"/engineering/pkce/"},"html":"

PKCE is an OAuth 2.0 security extension for public clients on mobile devices intended to avoid a malicious programme creeping into the same computer from intercepting the authorisation code. The RFC 7636 introduction discusses the mechanisms of such an attack.

\n

PKCE has a different specification of its own. It allows applications to use the most reliable OAuth 2.0 flows in public or untrusted clients - the Authorization Code flow. In order to efficiently use a dynamically generated password, it achieves this by doing some setup work before the flow and some verification at the end of the flow.

\n

This is important because getting a fixed secret in a public client is not safe.

\n

In this blog, we will see how PKCE is useful in authorization code flow for OAuth and OIDC and how you can use this with your OAuth and OpenID Connect providers.

\n

What is PKCE

\n

Proof Key for Code Exchange as known as PKCE, is a key for preventing malicious attacks and securely performing code authorization flow.

\n

I would say, PKCE is used to provide one more security layer to the authorization code flow in OAuth and OpenID Connect.

\n

PKCE is mainly useful for the client-side application or any web apps that are using the client secret key and used to replace the static secret used in the authorization flow.

\n

This flow basically works with two parameters Code Verifier and Code challenge. Let's see what are these parameters, how we use them, and generate them.

\n

PKCE code verifier and challenge

\n

The code verifier is a cryptographically random string using the characters A-Z, a-z, 0-9, and the punctuation characters -._~ (hyphen, period, underscore, and tilde), between 43 and 128 characters long.\nOnce the client has generated the code verifier, it uses that to create the code challenge.

\n

For devices that can perform a SHA256 hash, the code challenge is a BASE64-URL-encoded string of the SHA256 hash of the code verifier.

\n

Generate code verifier and code challenge

\n

Here you can see the examples to generate the Code verifier and code challenge in different languages. Either you can find Node and Go Packages for this but I would recommend you to not depend on any package for such small things.

\n

NodeJs

\n
var crypto = require("crypto")\n\nfunction base64URLEncode(str) {\n    return str.toString('base64')\n        .replace(/\\+/g, '-')\n        .replace(/\\//g, '_')\n        .replace(/=/g, '');\n}\nvar verifier = base64URLEncode(crypto.randomBytes(32));\nconsole.log("code_verifier: ", verifier)\n\nif(verifier){\n    var challenge = base64URLEncode(sha256(verifier));\n    console.log("code_challenge: ",challenge)\n}\n\n\nfunction sha256(buffer) {\n    return crypto.createHash('sha256').update(buffer).digest();\n}
\n

Golang

\n
package main\n \nimport (\n    "crypto/sha256"\n    "encoding/base64"\n    "fmt"\n    "math/rand"\n    "strings"\n    "time"\n)\n \ntype CodeVerifier struct {\n    Value string\n}\n \nconst (\n    length = 32\n)\n \nfunc base64URLEncode(str []byte) string {\n    encoded := base64.StdEncoding.EncodeToString(str)\n    encoded = strings.Replace(encoded, "+", "-", -1)\n    encoded = strings.Replace(encoded, "/", "_", -1)\n    encoded = strings.Replace(encoded, "=", "", -1)\n    return encoded\n}\n \nfunc verifier() (*CodeVerifier, error) {\n    r := rand.New(rand.NewSource(time.Now().UnixNano()))\n    b := make([]byte, length, length)\n    for i := 0; i < length; i++ {\n        b[i] = byte(r.Intn(255))\n    }\n    return CreateCodeVerifierFromBytes(b)\n}\n \nfunc CreateCodeVerifierFromBytes(b []byte) (*CodeVerifier, error) {\n    return &CodeVerifier{\n        Value: base64URLEncode(b),\n    }, nil\n}\n \nfunc (v *CodeVerifier) CodeChallengeS256() string {\n    h := sha256.New()\n    h.Write([]byte(v.Value))\n    return base64URLEncode(h.Sum(nil))\n}\n \nfunc main() {\n    verifier, _ := verifier()\n    fmt.Println("code_verifier: ", verifier.Value)\n    challenge := verifier.CodeChallengeS256()\n    fmt.Println("code_challenge :", challenge)\n}\n
\n

Implement the OAuth 2.0 Authorization Code with PKCE Flow

\n

Get the Authorization code

\n

In the OAuth Authorization flow, we need to have the code verifier and code challenge to start with the authentication and obviously an OAuth provider to connect.

\n

For the initial request, we need to pass the codechallenge and codechallenge_method to the OAuth or OIDC provider that supports PKCE based flow.

\n

The request will look like:

\n
Provider + /oauth/redirect?\nclient_id={client_id}\n&redirect_uri={Callback URL}\n&scope={Scope}\n&response_type=code\n&state={random long string}\n&code_challenge={code challenge}\n&code_challenge_method=SHA256
\n

The provider should redirect you to the authentication/login page and where you’ll get the code after successful authentication.

\n

Code Exchange

\n

In the code exchange request, we need to pass the code we have received through the above request and the code verifier that we have generated in our first step.

\n
POST Provider + /oauth/access_token\n\nRequest body:\n{\n   client_id:{client_id},\n   client_secret:{client_secret},\n   redirect_uri:{redirect_uri},\n   response_type:token,\n   Code:{code} // That we have received in authorization request\n   code_verifier: {code verifier // generated in the first step\n}
\n

Once the codeverificer hash matches with the codechallenge of the authorization request, You will get the token in the response with status code 200 OK.

\n
{\n    "access_token": "c5a****b-****-4*7f-a********e4a",\n    "token_type": "access_token",\n    "refresh_token": "5*****82-b***-**82-8c*1-*******7ce",\n    "expires_in": 11972\n}
\n

That's it and you've implemented PKCE flow in your application.

\n","frontmatter":{"date":"September 03, 2020","updated_date":null,"description":"If you are working with OAuth and OIDC authorization code flow and want to setup PKCE flow then this article will help you to understand everything about PKCE.","title":"PKCE: What it is and how to use it with OAuth 2.0","tags":["PKCE","Oauth","OIDC"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/7522430985a36d7f8c6267ed773d5447/ad85c/pkce.webp","srcSet":"/static/7522430985a36d7f8c6267ed773d5447/61e93/pkce.webp 200w,\n/static/7522430985a36d7f8c6267ed773d5447/1f5c5/pkce.webp 400w,\n/static/7522430985a36d7f8c6267ed773d5447/ad85c/pkce.webp 600w","sizes":"(max-width: 600px) 100vw, 600px"}}},"author":{"id":"Narendra Pareek","github":"pareek-narendra","avatar":null}}}},{"node":{"excerpt":"Big Data - Introduction Earlier, we were only dealing with well-structured data, hosted in large data warehouses, and investing a cost in…","fields":{"slug":"/engineering/big-data-testing-strategy/"},"html":"

Big Data - Introduction

\n

Earlier, we were only dealing with well-structured data, hosted in large data warehouses, and investing a cost in maintaining those data warehouses and hiring expert professional to maintain and secure information hosted in that data warehouse. Data was structured and can be queried anything as per the needs. But now, this exponential growth of data generates a new vision for data science along with some major challenges.

\n

Big Data is something which is growing exponentially with time, and carry raw but very valuable information inside that can change the future of any enterprise. It is a collection, which represents a large dataset, may be collected from multiple sources, or stored in an organization.\nLet‟s understand a real-time example, Big companies like Ikea and Amazon are leveraging the benefit of big data by collecting data from customer‟s buying patterns at their stores, their internal stock information, and their inventory demand-supply relations and analyze all, in seconds even in real-time to add value to its customer experience.

\n

So, extracting information from a large dataset somewhat calls a concept of Data mining which is an analytic process originally designed to explore large datasets. The ultimate goal of data mining is to search for consistency in a pattern or systematic relationship between variables, which helps in predicting the next pattern or behavior.

\n

Now, if we take concepts of data mining forward along with large data set, to some extent it becomes a blocker for our existing approach, because big data may contain structured or unstructured data even it may contain data in multiple formats also.

\n

Big Data Testing

\n

Testing is an art of achieving quality in your software product, in terms of perfection of functionality, performance, user experience, or usability. But for big data testing, you need to keep your focus more on the functional and performance aspects of an application. Performance is the key parameter in any big data application which is meant to process terabytes of data. Successful processing of terabytes of data using a commodity cluster with a number of other supportive components needs to be verified. Processing should be faster and accurate which demands a high level of testing.

\n

Processing may be of three types:–

\n

\"Batch,

\n

And based on which, we need to integrate different components along with NoSQL data store as per the needs.

\n

Big Data Testing - Test Data

\n

Data plays a vital role in the testing of big data applications. Application is meant to process data and provide an expected output based on implemented logic. The logic needs to be verified before moving to production, as the implementation of logic is completely based on business requirements and data.

\n

1. Test Data Quality

\n

Good quality test data is as important as the test environment. In the big data world, data can have any format or size, it may be in the form of a document, XML, JSON, PDF, etc. at the same time data size may go up to terabytes of petabytes. Hence, test data should also have multiple formats and size should be large enough to ensure the handling of large data processing. In big data testing, it needs data with logical values as per the application requirement and format which is supported by the application.

\n

Along with it, data quality is another aspect of big data testing. Ensuring the quality of data before processing through application ensures the accuracy of the final output. Data quality testing itself is a huge domain and covers a lot of best practices which include – data completeness, conformity, accuracy, validity, duplication, and consistency, etc. It should be included in the big data testing and this ensures the level of accuracy application is supposed to provide.

\n

2. Test Data Generation

\n

The generation of test data is again a challenging job, there are multiple parameters, which have to be taken care of while generating test data. It needs a tool, which can help to generate data and should have functions or logic can also be applied over it. Tools like Talend (an open studio) is the best candidate to fulfill the requirements of data generation.

\n

3. Data Storage

\n

After the generation of test data along with quality, it needs to host on a file system. For testing big data applications, data should be stored in the system similar to the production environment. As we are working in big data space, there should have a different number of nodes, and data must be in a distributed environment.

\n

Big Data Testing - Test Environment

\n

In Big data testing, the test environment should be efficient enough to process a large amount of data as done in the case of a production environment. Real-time production environment clusters generally have 30-40 nodes of cluster and data is distributed on the cluster nodes. There must have some minimum configuration for each node used in the cluster. A cluster may have two modes, in-premise or cloud. For testing in big data, it needs the same kind of environment with some minimum configuration of node.

\n

Scalability is also desired to be there in the test environment of big data testing, it helps to study the performance of application with the increase in the number of resources. That data can be used to define SLA (service level agreement) for that particular application.

\n

Big Data Testing can be categorized into three stages:

\n

\"Big

\n

Step 1: Data Staging Validation

\n

The first stage of big data testing, also known as a Pre-Hadoop stage, is comprised of process validation.

\n
    \n
  1. Validation of data is very important so that the data collected from various source like RDBMS, weblogs, etc are verified and then added to the system.
    2. \n
  2. To ensure data match you should compare source data with the data added to the Hadoop system.
    3. \n
  3. Make sure that the right data is taken out and loaded into the accurate HDFS location
    4. \n
\n

Step 2: “Map Reduce” Validation

\n

Validation of “Map Reduce” is the second stage. Business logic validation on every node is performed by the tester. Post that authentication is done by running them against multiple nodes, to make sure that the:

\n\n

Step 3: Output Validation Phase

\n

The output validation process is the final or third stage involved in big data testing. The output data files are created and they are ready to be moved to an EDW (Enterprise Data Warehouse) or any other such system as per requirements. The third stage consisted of:

\n\n

Big Data - Performance Testing

\n

Big data applications are meant to process a large amount of data, and it is expected that it should take minimum time to process maximum data. Along with it, application jobs should consume a considerable amount of memory and CPU. In big data testing, performance parameter plays an important role and helps to define SLA's. It covers the performance of the base machine and cluster. Also, for example, In the case of Hadoop, map-reduce jobs should be written with proper coding guidelines, to perform better in the production environment. Profiling can also be done on map-reduce jobs before integration, to ensure their optimized execution.

\n

Tools used in Big Data Scenarios

\n

\"Big

\n

Big Data Testing - Challenges

\n

In big data testing, certain challenges are involved which needs to be addressed by the big data testing approach.

\n

1. Test Data

\n

Exponential growth had been observed in the growth of data in the last few years. A huge amount of data are being generated daily and stored in large data centers or data marts. So, there is a demand for efficient storage and a way to process it in an optimized way. If we consider the telecom industry, it generates a large number of call logs daily and they need to be processed for better customer experience and compete in the market. The same goes with the test data, test data should be similar to production data and should contain all the logically acceptable fields in it.

\n

This becomes a challenge for testing big data application, generating test data similar to production data is a real challenge. Test data should also be large enough to verify proper working big data application.

\n

2. Environment

\n

The processing of data highly depends on the environment and its performance. An optimized environment setup gives high performance and fast data processing results. Distributed computing is used for the processing of big data which has data hosted in a distributed environment. The testing environment should have multiple numbers of nodes and data should be distributed over the nodes. At the same time, it also needs to monitor those nodes, to ensure the highest performance with minimum CPU and memory utilization. Nodes should be monitored and there should have a graphical presentation of node performance. So, the test environment has two aspects – distributed nodes and their monitoring, which should be covered in the testing approach.

\n

3. Performance

\n

Performance is the key requirement of any big data application, and of course because of which enterprises are moving towards NoSQL technologies, technologies that can handle their big data and process in the minimum time frame. A large dataset should be processed in a minimum considerable time frame. In big data testing, performance testing is a challenge, it requires monitoring of cluster nodes during execution and also time is taken for every iteration of execution.

\n

Conclusion

\n

Big Data is the trend that is revolutionizing society and its organizations due to the capabilities it provides to take advantage of a wide variety of data, in large volumes and with speed. Keeping the challenges in mind we have defined the approach of testing big data applications. This approach of big data testing will make it easy for a test engineer to verify and certify the business requirement implementations and for stack holders, it saves a huge amount of cost, which has to be invested to get the expected business returns.

\n","frontmatter":{"date":"September 02, 2020","updated_date":null,"description":"With the exponential growth in the number of big data applications in the world, Testing in big data applications is related to database, infrastructure and performance testing and functional testing. This blog guides what should be the strategy for testing Big Data applications.","title":"Big Data - Testing Strategy","tags":["Testing","Big Data","Big Data Testing"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7699115044247788,"src":"/static/2997b1b5bea2cc94a828fae181ebf972/58556/big-data.webp","srcSet":"/static/2997b1b5bea2cc94a828fae181ebf972/61e93/big-data.webp 200w,\n/static/2997b1b5bea2cc94a828fae181ebf972/1f5c5/big-data.webp 400w,\n/static/2997b1b5bea2cc94a828fae181ebf972/58556/big-data.webp 800w,\n/static/2997b1b5bea2cc94a828fae181ebf972/99238/big-data.webp 1200w,\n/static/2997b1b5bea2cc94a828fae181ebf972/7c22d/big-data.webp 1600w,\n/static/2997b1b5bea2cc94a828fae181ebf972/25f09/big-data.webp 1920w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Sudhey Sharma","github":"sudheysharma","avatar":null}}}},{"node":{"excerpt":"Introduction Authentication is the necessary implementation for secure and genuine access to resources of web and mobile applications. It…","fields":{"slug":"/engineering/email-verification-api/"},"html":"

Introduction

\n

Authentication is the necessary implementation for secure and genuine access to resources of web and mobile applications. It helps to authorize the user and provide access control for applications based on the user’s credentials.

\n

There are various authentication mechanisms like Email, Phone numbers, Biometrics, etc. However, Email is still considered the primary means of authentication because other mechanisms are not as optimized or in approach to everyone.

\n

So it is safe to say that most of your application users used email for creating their accounts. With that, some users tend to use disposable email services for fake account creation. These fake accounts result in additional liability to your application resources and servers.

\n

You can effectively reduce such spams for your applications by analyzing the user’s email address during registration for the following aspects:

\n\n

There are some tools available that could help you analyze the above factors, and one such tool is EVA, an Email Verification API. It is a free API and analyzes the user’s email for all of the factors mentioned above.

\n

API Details

\n

The details of Email Verification API (API) is as follows:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ParametersValue
URLhttps://api.eva.pingutil.com/email
Query Param{ email: test@mail7.io }
Response{ \"status\": \"success\", \"data\": { \"email_address\": \"test@mail7.io\", \"domain\": \"mail7.io\", \"valid_syntax\": true, \"disposable\": true, \"deliverable\": true }}
\n

Valid_syntax, disposable, and deliverable are three attributes of this API, let’s see how these works:

\n

Output Parameters:

\n

valid_syntax\nThis attribute verifies the syntax of the entered email address matches the standard email address format or not. It is useful in the cases where the end-user might misspell email addresses while typing.

\n

disposable\nThis attribute verifies whether the entered email address domain belongs to the disposable email service. EVA checks the email address against a valid database of more than 4500 disposable email services. Thus, it can effectively help the developer to prevent users from registering using disposable email.

\n

deliverable\nThis attribute verifies whether the email domain has valid MX records or not. The email address is considered deliverable if the valid MX record for the email domain is found.

\n
\n

Note: A mail exchanger record (MX record) specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS).

\n
\n

The response received for the above three attributes of API can help you take further action. These attributes are not just limited to registration use cases, you can utilize them for other use cases like if you have an existing user base but would like to limit the spammers (who have registered using disposable emails) from commenting on the blog or accessing your application resources.

\n

This API tool has been developed and designed as a way of giving back to the developer community initiative by LoginRadius Developers. Your opinion is important to us. This way we can keep improving our product for the developer community. Please share your feedback here

\n","frontmatter":{"date":"August 31, 2020","updated_date":null,"description":"Email authentication, being the traditional way of authentication and used most widely. Increase in the spams has also increased the disposable email registrations. To reduce and identify such unwanted users, EVA(Email Verification API) is the tool, developed by LoginRadius developers.","title":"Email Verification API (EVA)","tags":["Free tool","Developer Resources ","Verification","Email"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7699115044247788,"src":"/static/9674b13085ba4d36812d596ecc764c38/58556/eva.webp","srcSet":"/static/9674b13085ba4d36812d596ecc764c38/61e93/eva.webp 200w,\n/static/9674b13085ba4d36812d596ecc764c38/1f5c5/eva.webp 400w,\n/static/9674b13085ba4d36812d596ecc764c38/58556/eva.webp 800w,\n/static/9674b13085ba4d36812d596ecc764c38/99238/eva.webp 1200w,\n/static/9674b13085ba4d36812d596ecc764c38/135cd/eva.webp 1280w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Aman Agrawal","github":"aman-agrwl","avatar":null}}}}]},"markdownRemark":{"excerpt":"Introduction Ever wondered how apps like Spotify, Netflix, or Slack manage seamless login experiences across devices? Many of them use JWT…","fields":{"slug":"/engineering/how-to-integrate-jwt/"},"html":"

Introduction

\n

Ever wondered how apps like Spotify, Netflix, or Slack manage seamless login experiences across devices? Many of them use JWT, or JSON Web Tokens, a compact, stateless method for securely transmitting user identity and session data across services.

\n

With JWT token authentication, identity information is embedded in a signed token, allowing you to maintain user sessions without server-side storage. This approach is highly scalable and ideal for modern architectures like SPAs, mobile apps, and microservices.

\n

In this blog, we’ll walk you through what is JWT, why use it, and how to implement JWT authentication using LoginRadius.

\n

You’ll learn what JWT is, why it’s effective, and how it works in real-world applications. We'll cover both integration methods (IDX and Direct API), generating your signing key, managing sessions, storing the JWT token securely, and applying best practices throughout.

\n

Whether you're a developer, product manager, or IAM architect, this guide offers a complete foundation for implementing JWT token authentication into your application stack.

\n

What is JWT?

\n

JSON Web Token (JWT) is an open standard (RFC 7519) used to transmit information securely between parties as a JSON object. It’s compact, self-contained, and digitally signed, making it a reliable format for authentication and authorization across modern applications.

\n

A JWT consists of three parts:

\n
    \n
  1. Header – Contains metadata like the type of token and signing algorithm (e.g., HS256).
    2. \n
  2. Payload – Stores the actual data or “claims,” such as user ID, roles, and token expiry.
    3. \n
  3. Signature – A cryptographic hash that ensures the token hasn’t been tampered with.
    4. \n
\n

Example of a token structure:

\n

<base64Header>.<base64Payload>.<signature>

\n

Why Use JWT?

\n\n

How JWT Works?

\n

\"Flowchart

\n
    \n
  1. A user logs in with credentials.
    2. \n
  2. Your app (or identity provider like LoginRadius) issues a signed JWT.
    3. \n
  3. The client stores the token and sends it with each request (usually in the Authorization header).
    4. \n
  4. The server validates the token’s signature and claims.
    5. \n
  5. If valid, access is granted — without any session stored on the backend.
    6. \n
\n

JWT simplifies identity verification, especially when you're building apps that talk to APIs or need to scale without centralized session storage.

\n

JWT Authentication with LoginRadius: Overview

\n

LoginRadius provides robust support for JWT (JSON Web Token) authentication, which allows for flexible and secure access control across different digital platforms. Whether you're building a fully custom identity flow or using a pre-built interface, the platform supports various integration approaches depending on your architecture.

\n

If you're looking to understand how to implement JWT token authentication effectively, LoginRadius offers two primary implementation models that cater to different levels of customization and control:

\n

1. IDX Implementation – JWT through a Hosted Login Page

\n

The IDX-hosted login approach enables secure, standards-compliant, JWT-based authentication without requiring you to build a custom login interface. This is a strategic option for fast, compliant, and user-friendly deployments.

\n\n

Configuration Steps:

\n
    \n
  1. Enable JWT Login
    2. \n
  2. Go to authentication configuration settings and enable JWT Login in the Admin Console.
    3. \n
\n

\"Screenshot

\n
    \n
  1. Specify your signing algorithm and expiry policy, and define your JWT Secret Key.
    2. \n
  2. Input a secure JWT signing key.
    3. \n
  3. Specify token expiry duration (e.g., 15–60 minutes)
    4. \n
  4. Select the desired algorithm —HS256 for symmetric signing (same key signs and verifies)
    5. \n
  5. RS256 for asymmetric signing, where LoginRadius securely stores the private key used to sign the JWT.
    6. \n
  6. Your app or backend service uses the public key to validate the token signature.
    7. \n
  7. LoginRadius provides a JWKS (JSON Web Key Set) endpoint to dynamically fetch and rotate public keys, ensuring trust without key exposure.
    8. \n
  8. Update IDX Template for Callback
    9. \n
  9. Modify your IDX login page template to retrieve the JWT post-login. You can access the token via redirect URL parameters or secure JavaScript callbacks.
    10. \n
\n

Example Response:

\n

{

\n

\"access_token\": \"eyJhbGciOiJIUzI1NiIsInR...\",

\n

\"expires_in\": 1800

\n

}

\n

This integration approach works best for all teams that want effective identity workflows without the complexity of building proprietary login screens, something that is crucial for customer portals, onboarding of mobile applications, and even managing access for business partners.

\n

2. Direct API Implementation – Self Managed Login

\n

If you’re building a custom login UI or working in a headless environment, LoginRadius lets you generate and handle JWTs directly through its Authentication APIs. Here’s how you can programmatically perform token authentication using the classic method:

\n\n

Configuration Steps:

\n

Step 1: Authenticate via API:

\n\n

Include the user’s credentials (email + password) in the request body.

\n

Step 2: Get JWT in Response

\n\n

{

\n

\"access_token\": \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...\",

\n

\"expires_in\": 3600

\n

}

\n

Step 3: JWT Decoding and Validation

\n\n

Step 4: Set Custom Claims (Optional)

\n

With LoginRadius, it is possible to customize the payload to include user roles and/or any additional metadata. You can set custom JWT claims on the Admin Console.

\n

With this method, you have complete customization over login flows while using LoginRadius to issue signed JWTs for user session management.

\n

NOTE- With either method, LoginRadius ensures that JWTs are securely signed, optionally short-lived, and compatible with standard token validation libraries, making integration seamless for everyone.

\n

To get started with JWT implementation, you can read our complete developer documentation.

\n

Hosted Login vs Direct API

\n

\"Illustration

\n

What is Session Management and How It Works with JWT

\n

Session management is how your app keeps track of a user after they log in so they don’t have to prove who they are with every request.

\n

In traditional apps, sessions are stored on the server using session IDs. Every time a request comes in, the server checks that session ID to verify the user.

\n

In modern apps, especially SPAs and APIs, JWTs are used to manage sessions without needing server-side storage; this is called stateless session management. The token itself carries the user’s identity, roles, and expiration details. As long as the token is valid, the user stays logged in.

\n

Good session management ensures:

\n\n

How LoginRadius Handles Session Management with JWT:

\n
    \n
  1. \n

    User Logs In

    \n\n
    2. \n
  2. \n

    Client Stores the Token

    \n\n
    3. \n
  3. \n

    Access Token Expiry

    \n\n
    4. \n
  4. \n

    Token Renewal

    \n\n
    5. \n
  5. Logout and Token Revocation Strategy
    6. \n
\n

When the user logs out, both the access token and refresh token should be cleared from client storage.

\n\n

Combining these strategies gives you greater control over token misuse and enables a robust, enterprise-grade logout flow.

\n

\"illustration

\n

How to Store JWT Tokens?

\n

When you implement JWT-based authentication, the client (browser or mobile app) needs a way to store the access token and, optionally, the refresh token after they are issued by the authentication server. This stored token is then attached to every subsequent request to prove the user's identity.

\n

Choosing where to store the JWT is a crucial security decision. The most common storage options are:

\n\n

Each option has trade-offs between security, accessibility, and persistence, and the right choice depends on your application's architecture and threat model.

\n

Recommended Storage Strategy

\n\n

Best Practices for Storing JWTs

\n
    \n
  1. Never store sensitive tokens (like refresh tokens) in localStorage or sessionStorage.
    2. \n
  2. Use Secure and HttpOnly flags with cookies to prevent JavaScript access and ensure transmission only over HTTPS.
    3. \n
  3. Set the SameSite=Strict or Lax attribute on cookies to protect against CSRF.
    4. \n
  4. Use short-lived access tokens and rotate refresh tokens regularly.
    5. \n
  5. Implement CSP (Content Security Policy) to reduce XSS risk.
    6. \n
  6. Avoid storing any tokens in frontend code (e.g., hardcoded in JS files).
    7. \n
\n

Conclusion

\n

JWT authentication with LoginRadius offers a modern, stateless approach to managing sessions across distributed systems. The IDX integration is ideal for rapid deployment, while the Direct API model is best for organizations needing deep customization and integration flexibility.

\n

With robust token signing, refresh capabilities, and centralized control, LoginRadius provides a future-ready foundation for secure, scalable identity architecture. Contact us to know more about JWT authentication and implementation guide.

\n

FAQs

\n

1. What is JWT authentication used for?

\n

A: JWT authentication securely verifies user identities, enabling stateless session management across web, mobile apps, and microservices without server-side session storage.

\n

2. How does LoginRadius simplify JWT integration?

\n

A: LoginRadius simplifies JWT integration by offering hosted IDX login pages and direct API-based authentication methods, enabling rapid deployment and deep customization.

\n

3. Is JWT authentication secure?

\n

A: Yes, JWT authentication is secure when implemented with best practices like short-lived tokens, secure storage methods, signature validation, and refresh token rotation.

\n

4. Can JWT tokens be revoked with LoginRadius?

\n

A: Yes, LoginRadius allows revocation of JWT refresh tokens explicitly, and supports strategies like short-lived tokens and key rotation to manage token lifecycles securely.

\n","frontmatter":{"date":"April 15, 2025","updated_date":null,"description":"Discover JWT (JSON Web Token) authentication, its advantages, and how to integrate it seamlessly using LoginRadius' hosted IDX and Direct API methods for secure, scalable identity management.","title":"JWT Authentication with LoginRadius: Quick Integration Guide","tags":["JWT","JSON Web Token","Authentication","Authorization"],"pinned":null,"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":0.7782101167315175,"src":"/static/4cedb7829f98208cbc6d5a9aea4e983d/58556/how-to-integrate-jwt.webp","srcSet":"/static/4cedb7829f98208cbc6d5a9aea4e983d/61e93/how-to-integrate-jwt.webp 200w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/1f5c5/how-to-integrate-jwt.webp 400w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/58556/how-to-integrate-jwt.webp 800w,\n/static/4cedb7829f98208cbc6d5a9aea4e983d/1cc9f/how-to-integrate-jwt.webp 896w","sizes":"(max-width: 800px) 100vw, 800px"}}},"author":{"id":"Kundan Singh","github":null,"avatar":null}}}},"pageContext":{"limit":6,"skip":174,"currentPage":30,"type":"//engineering//","numPages":53,"pinned":"5c425581-f474-5ae9-abe7-cf5342db2aaa"}},"staticQueryHashes":["1171199041","1384082988","2100481360","23180105","528864852"]}