#memory-management
#memory-management

[ follow ]

TrapC proposal to fix C/C++ memory safety

TrapC enhances C's capabilities by improving safety and memory management while maintaining C's minimalist design.

Modeling Complex Data Structure in Golang Using Pointers, References, and Reverse Indices

Go pointers and references are essential for building efficient systems in the Go language.

Understanding pointers can be complicated, but practical projects can aid learning.

A Simplified Comparison: Rust and Pointers | HackerNoon

Rust ensures memory safety through its unique ownership and borrowing model, mitigating risks present in traditional languages.

TrapC proposal to fix C/C++ memory safety

TrapC enhances C's capabilities by improving safety and memory management while maintaining C's minimalist design.

Modeling Complex Data Structure in Golang Using Pointers, References, and Reverse Indices

Go pointers and references are essential for building efficient systems in the Go language.

Understanding pointers can be complicated, but practical projects can aid learning.

A Simplified Comparison: Rust and Pointers | HackerNoon

Rust ensures memory safety through its unique ownership and borrowing model, mitigating risks present in traditional languages.

moreprogramming-languages

#python

The surprising way to save memory with BytesIO

Efficient data handling in Python's BytesIO can save memory and improve performance.

Memoryview allows access to data without creating duplicates, preserving memory.

Catching memory leaks with your test suite

Resource leaks cause programs to gradually consume more limited resources, leading to crashes.

The surprising way to save memory with BytesIO

Efficient data handling in Python's BytesIO can save memory and improve performance.

Memoryview allows access to data without creating duplicates, preserving memory.

Catching memory leaks with your test suite

Resource leaks cause programs to gradually consume more limited resources, leading to crashes.

morepython

#data-processing

How to Split a Python List or Iterable Into Chunks - Real Python

Splitting a long list into fixed-size chunks can enhance performance and manageability in programming and data transfer.

Optimizing Resource Allocation and Parallel Processing for 20GB Spark Jobs

Optimizing resource allocation based on data volume and processing speed is crucial for efficient job completion.

How to Split a Python List or Iterable Into Chunks - Real Python

Splitting a long list into fixed-size chunks can enhance performance and manageability in programming and data transfer.

Optimizing Resource Allocation and Parallel Processing for 20GB Spark Jobs

Optimizing resource allocation based on data volume and processing speed is crucial for efficient job completion.

moredata-processing

#software-engineering

Challenges of Creating iOS App Extensions at Lyft

Lyft engineers efficiently manage iOS app extension development by optimizing dependencies, binary size, and memory usage while adhering to Apple's constraints.

RRR-Rust, RAR, and RAC: The Trifecta for Co-Design Excellence | HackerNoon

Rust offers improved type safety and memory management through its ownership model.

Restricted Algorithmic Rust serves as a bridge from C/C++ to RAC while maintaining Rust's proper syntax.

Challenges of Creating iOS App Extensions at Lyft

Lyft engineers efficiently manage iOS app extension development by optimizing dependencies, binary size, and memory usage while adhering to Apple's constraints.

RRR-Rust, RAR, and RAC: The Trifecta for Co-Design Excellence | HackerNoon

Rust offers improved type safety and memory management through its ownership model.

Restricted Algorithmic Rust serves as a bridge from C/C++ to RAC while maintaining Rust's proper syntax.

moresoftware-engineering

How I Solved the WebSocket Scaling Problem Without Breaking the Bank | HackerNoon

WebSockets enable scalable systems for handling many connections efficiently with proper connection management.

#pagedattention

Evaluating vLLM With Basic Sampling | HackerNoon

vLLM outperforms other models in handling higher request rates while maintaining low latencies through efficient memory management.

How vLLM Implements Decoding Algorithms | HackerNoon

vLLM optimizes large language model serving through innovative memory management and GPU techniques.

PagedAttention: An Attention Algorithm Inspired By the Classical Virtual Memory in Operating Systems | HackerNoon

PagedAttention optimizes memory usage in language model serving, significantly improving throughput while minimizing KV cache waste.

How Good Is PagedAttention at Memory Sharing? | HackerNoon

Memory sharing in PagedAttention enhances efficiency in LLMs, significantly reducing memory usage during sampling and decoding processes.

Our Method for Developing PagedAttention | HackerNoon

PagedAttention optimizes memory usage in LLM serving by managing key-value pairs in a non-contiguous manner.

Evaluating vLLM's Design Choices With Ablation Experiments | HackerNoon

PagedAttention significantly enhances vLLM's performance despite adding overhead, illustrating the trade-offs in optimizing GPU operations for large language models.

Evaluating vLLM With Basic Sampling | HackerNoon

vLLM outperforms other models in handling higher request rates while maintaining low latencies through efficient memory management.

How vLLM Implements Decoding Algorithms | HackerNoon

vLLM optimizes large language model serving through innovative memory management and GPU techniques.

PagedAttention: An Attention Algorithm Inspired By the Classical Virtual Memory in Operating Systems | HackerNoon

PagedAttention optimizes memory usage in language model serving, significantly improving throughput while minimizing KV cache waste.

How Good Is PagedAttention at Memory Sharing? | HackerNoon

Memory sharing in PagedAttention enhances efficiency in LLMs, significantly reducing memory usage during sampling and decoding processes.

Our Method for Developing PagedAttention | HackerNoon

PagedAttention optimizes memory usage in LLM serving by managing key-value pairs in a non-contiguous manner.

Evaluating vLLM's Design Choices With Ablation Experiments | HackerNoon

PagedAttention significantly enhances vLLM's performance despite adding overhead, illustrating the trade-offs in optimizing GPU operations for large language models.

morepagedattention

#large-language-models

Practical LLMs for Real-World Applications | HackerNoon

Anchor-based LLMs reduce memory use by 99% while improving inference speed by up to 3.5 times, enabling practical use on resource-constrained devices.

How Effective is vLLM When a Prefix Is Thrown Into the Mix? | HackerNoon

vLLM significantly improves throughput in LLM tasks by utilizing shared prefixes among different input prompts.

Practical LLMs for Real-World Applications | HackerNoon

Anchor-based LLMs reduce memory use by 99% while improving inference speed by up to 3.5 times, enabling practical use on resource-constrained devices.

How Effective is vLLM When a Prefix Is Thrown Into the Mix? | HackerNoon

vLLM significantly improves throughput in LLM tasks by utilizing shared prefixes among different input prompts.

morelarge-language-models

#transformer-models

Evaluating the Performance of vLLM: How Did It Do? | HackerNoon

vLLM was tested using various Transformer-based large language models to evaluate its performance under load.

The Generation and Serving Procedures of Typical LLMs: A Quick Explanation | HackerNoon

Transformer-based language models use autoregressive approaches for token sequence probability modeling.

Batching Techniques for LLMs | HackerNoon

Batching improves compute utilization for LLMs, but naive strategies can cause delays and waste resources. Fine-grained batching techniques offer a solution.

Evaluating the Performance of vLLM: How Did It Do? | HackerNoon

vLLM was tested using various Transformer-based large language models to evaluate its performance under load.

The Generation and Serving Procedures of Typical LLMs: A Quick Explanation | HackerNoon

Transformer-based language models use autoregressive approaches for token sequence probability modeling.

Batching Techniques for LLMs | HackerNoon

Batching improves compute utilization for LLMs, but naive strategies can cause delays and waste resources. Fine-grained batching techniques offer a solution.

moretransformer-models

#vllm

The Distributed Execution of vLLM | HackerNoon

Large Language Models often exceed single GPU limits, requiring advanced distributed execution techniques for memory management.

How vLLM Prioritizes a Subset of Requests | HackerNoon

vLLM utilizes FCFS scheduling and an all-or-nothing eviction policy to effectively manage resources and prioritize fairness in request handling.

The Distributed Execution of vLLM | HackerNoon

Large Language Models often exceed single GPU limits, requiring advanced distributed execution techniques for memory management.

How vLLM Prioritizes a Subset of Requests | HackerNoon

vLLM utilizes FCFS scheduling and an all-or-nothing eviction policy to effectively manage resources and prioritize fairness in request handling.

morevllm

#deep-learning

Pytorch Contiguous Tensor Optimization | HackerNoon

Efficient memory management and tensor contiguity are essential for optimizing performance in PyTorch, especially when handling large-scale datasets.

PagedAttention: Memory Management in Existing Systems | HackerNoon

Current LLM serving systems inefficiently manage memory, resulting in significant waste due to fixed size allocations based on potential maximum sequence lengths.

Pytorch Contiguous Tensor Optimization | HackerNoon

Efficient memory management and tensor contiguity are essential for optimizing performance in PyTorch, especially when handling large-scale datasets.

PagedAttention: Memory Management in Existing Systems | HackerNoon

Current LLM serving systems inefficiently manage memory, resulting in significant waste due to fixed size allocations based on potential maximum sequence lengths.

moredeep-learning

#app-compatibility

Android 15 Beta 4 Now Available for Developers to Bring their Apps Up to Date

The final Android 15 beta focuses on stable APIs for developer use and introduces significant behavior changes and new privacy features.

Google urges Android developers to prep for 16 KB memory page

Android developers must prepare for a 16 KB memory page size upgrade to gain performance benefits of 5-10% in apps and games.

Android 15 Beta 4 Now Available for Developers to Bring their Apps Up to Date

The final Android 15 beta focuses on stable APIs for developer use and introduces significant behavior changes and new privacy features.

Google urges Android developers to prep for 16 KB memory page

Android developers must prepare for a 16 KB memory page size upgrade to gain performance benefits of 5-10% in apps and games.

moreapp-compatibility

#scala

How We Saved 12% in Resources with Smarter Heap Management

Optimizing buffer sizes and investigating memory usage is crucial for performance in Scala services to prevent Out of Memory errors.

How We Saved 12% in Resources with Smarter Heap Management

Memory issues in a Scala service stemmed from underutilized buffers in Netty and inefficient memory management by the JSON library Jsoniter.

How We Saved 12% in Resources with Smarter Heap Management

Memory issues in Scala service were traced to buffer mismanagement, leading to high memory usage and frequent GC runs.

Optimized buffer sizes and investigated long-lived object handling to address memory challenges.

How We Saved 12% in Resources with Smarter Heap Management

Optimizing buffer sizes and investigating memory usage is crucial for performance in Scala services to prevent Out of Memory errors.

How We Saved 12% in Resources with Smarter Heap Management

Memory issues in a Scala service stemmed from underutilized buffers in Netty and inefficient memory management by the JSON library Jsoniter.

How We Saved 12% in Resources with Smarter Heap Management

Memory issues in Scala service were traced to buffer mismanagement, leading to high memory usage and frequent GC runs.

Optimized buffer sizes and investigated long-lived object handling to address memory challenges.

morescala

LLM Service & Autoregressive Generation: What This Means | HackerNoon

LLMs generate tokens sequentially, relying on cached key and value vectors from prior tokens for efficient autoregressive generation.

Handling memory leaks in Rust - LogRocket Blog

Rust's ownership and borrowing principles help manage memory but memory leaks can still occur, necessitating careful management by developers.

Java 24 to Reduce Object Header Size and Save Memory

JEP 450 optimizes Java heap management by implementing compact object headers, which reduces header size and improves memory efficiency.

Chatbot Memory: Implement Your Own Algorithm From Scratch | HackerNoon

Implementing effective memory management is crucial for chatbot development, ensuring fluid and coherent interactions during long conversations.

Chrome introduces new 'Performance' tools to wrangle the tabs gobbling up your memory

Google introduces new Chrome features for better tab management through performance alerts and enhanced Memory Saver options.

An Efficient C++ Fixed Block Memory Allocator

Custom fixed block memory allocators improve memory management efficiency and reduce fragmentation issues, enhancing performance in critical and long-running systems.

Plumbing Life's Depths - Interesting Memory Leak with Python 3.12 for PyOpenGL-accelerate

The PyOpenGL 3.12 test suite's memory leak test fails uniquely on the 76th iteration, suggesting changes in Python's memory management.

#performance-optimization

Improve the performance of your Java application by using these optimizations

Optimize string concatenation with StringBuilder or StringBuffer.

Use local variables for frequently accessed data.

Optimize loops by moving invariant calculations outside.

Use switch statements for better performance.

Understanding Spark Re-Partition

Spark's repartition() function is crucial for managing data skewness, optimizing performance, memory utilization, and downstream query efficiency.

Improve the performance of your Java application by using these optimizations

Optimize string concatenation with StringBuilder or StringBuffer.

Use local variables for frequently accessed data.

Optimize loops by moving invariant calculations outside.

Use switch statements for better performance.

Understanding Spark Re-Partition

Spark's repartition() function is crucial for managing data skewness, optimizing performance, memory utilization, and downstream query efficiency.

moreperformance-optimization

#data-structures

Augmented Linked Lists: An Essential Guide | HackerNoon

Linked lists are efficient for fast addition of data without resizing the entire array, suitable for write-only data, and organizing data for sequential reads.

Augmented Tree Data Structures | HackerNoon

Data structures are key to efficient data storage and organization, crucial for memory management and optimizing software performance.

Augmented Linked Lists: An Essential Guide | HackerNoon

Linked lists are efficient for fast addition of data without resizing the entire array, suitable for write-only data, and organizing data for sequential reads.

Augmented Tree Data Structures | HackerNoon

Data structures are key to efficient data storage and organization, crucial for memory management and optimizing software performance.

moredata-structures

CISA Report Finds Most Open-Source Projects Contain Memory-Unsafe Code

More than half of critical open-source projects contain memory-unsafe code, leading to vulnerabilities like buffer overflows and memory leaks.

Kubernetes 1.30 Released with Contextual Logging, Improved Performance, and Security

Kubernetes 1.30 introduces improvements like memory swap support, sleep action for PreStop hook, and CEL for admission control.

Enhancements include beta support for user namespaces, more secure service account tokens, and Contextual Logging for better troubleshooting.

Scheduling improvements in 1.30 feature MatchLabelKeys for PodAffinity and PodAntiAffinity, enhancing pod placement strategies.

Linux free memory: How to show the free memory on a Linux system

You can show free memory on a Linux system using the free command with different parameters like -m for MB.

The top utility provides a real-time view of memory use by running applications, and the Linux ps command can be sorted by memory use.

How to control Java heap size (memory) allocation (xmx, xms)

Use -Xmx to specify maximum heap size and control RAM use in Java programs.

[ Load more ]

#memory-management#memory-management

PyCoder's Weekly | Issue #665

Go 1.24 Brings Generic Type Aliases, Weak Pointers, Improved Finalizers, and More

What You Have to Know About the New Features in Go 1.24: Cleanups and Weak Pointers | HackerNoon

PyCoder's Weekly | Issue #665

Go 1.24 Brings Generic Type Aliases, Weak Pointers, Improved Finalizers, and More

What You Have to Know About the New Features in Go 1.24: Cleanups and Weak Pointers | HackerNoon

Subverting control with weak references

ECMAScript proposal: Symbols as WeakMap keys

Subverting control with weak references

ECMAScript proposal: Symbols as WeakMap keys

TrapC proposal to fix C/C++ memory safety

Modeling Complex Data Structure in Golang Using Pointers, References, and Reverse Indices

A Simplified Comparison: Rust and Pointers | HackerNoon

TrapC proposal to fix C/C++ memory safety

Modeling Complex Data Structure in Golang Using Pointers, References, and Reverse Indices

A Simplified Comparison: Rust and Pointers | HackerNoon

The surprising way to save memory with BytesIO

Catching memory leaks with your test suite

The surprising way to save memory with BytesIO

Catching memory leaks with your test suite

How to Split a Python List or Iterable Into Chunks - Real Python

Optimizing Resource Allocation and Parallel Processing for 20GB Spark Jobs

How to Split a Python List or Iterable Into Chunks - Real Python

Optimizing Resource Allocation and Parallel Processing for 20GB Spark Jobs

Challenges of Creating iOS App Extensions at Lyft

RRR-Rust, RAR, and RAC: The Trifecta for Co-Design Excellence | HackerNoon

Challenges of Creating iOS App Extensions at Lyft

RRR-Rust, RAR, and RAC: The Trifecta for Co-Design Excellence | HackerNoon

How I Solved the WebSocket Scaling Problem Without Breaking the Bank | HackerNoon

Evaluating vLLM With Basic Sampling | HackerNoon

How vLLM Implements Decoding Algorithms | HackerNoon

PagedAttention: An Attention Algorithm Inspired By the Classical Virtual Memory in Operating Systems | HackerNoon

How Good Is PagedAttention at Memory Sharing? | HackerNoon

Our Method for Developing PagedAttention | HackerNoon

Evaluating vLLM's Design Choices With Ablation Experiments | HackerNoon

Evaluating vLLM With Basic Sampling | HackerNoon

How vLLM Implements Decoding Algorithms | HackerNoon

PagedAttention: An Attention Algorithm Inspired By the Classical Virtual Memory in Operating Systems | HackerNoon

How Good Is PagedAttention at Memory Sharing? | HackerNoon

Our Method for Developing PagedAttention | HackerNoon

Evaluating vLLM's Design Choices With Ablation Experiments | HackerNoon

Practical LLMs for Real-World Applications | HackerNoon

How Effective is vLLM When a Prefix Is Thrown Into the Mix? | HackerNoon

Practical LLMs for Real-World Applications | HackerNoon

How Effective is vLLM When a Prefix Is Thrown Into the Mix? | HackerNoon

Evaluating the Performance of vLLM: How Did It Do? | HackerNoon

The Generation and Serving Procedures of Typical LLMs: A Quick Explanation | HackerNoon

Batching Techniques for LLMs | HackerNoon

Evaluating the Performance of vLLM: How Did It Do? | HackerNoon

The Generation and Serving Procedures of Typical LLMs: A Quick Explanation | HackerNoon

Batching Techniques for LLMs | HackerNoon

The Distributed Execution of vLLM | HackerNoon

How vLLM Prioritizes a Subset of Requests | HackerNoon

The Distributed Execution of vLLM | HackerNoon

How vLLM Prioritizes a Subset of Requests | HackerNoon

Pytorch Contiguous Tensor Optimization | HackerNoon

PagedAttention: Memory Management in Existing Systems | HackerNoon

Pytorch Contiguous Tensor Optimization | HackerNoon

PagedAttention: Memory Management in Existing Systems | HackerNoon

Android 15 Beta 4 Now Available for Developers to Bring their Apps Up to Date

Google urges Android developers to prep for 16 KB memory page

Android 15 Beta 4 Now Available for Developers to Bring their Apps Up to Date

Google urges Android developers to prep for 16 KB memory page

How We Saved 12% in Resources with Smarter Heap Management

How We Saved 12% in Resources with Smarter Heap Management

How We Saved 12% in Resources with Smarter Heap Management

How We Saved 12% in Resources with Smarter Heap Management

How We Saved 12% in Resources with Smarter Heap Management

How We Saved 12% in Resources with Smarter Heap Management

LLM Service & Autoregressive Generation: What This Means | HackerNoon

Handling memory leaks in Rust - LogRocket Blog

Java 24 to Reduce Object Header Size and Save Memory

Chatbot Memory: Implement Your Own Algorithm From Scratch | HackerNoon

Chrome introduces new 'Performance' tools to wrangle the tabs gobbling up your memory

An Efficient C++ Fixed Block Memory Allocator

Plumbing Life's Depths - Interesting Memory Leak with Python 3.12 for PyOpenGL-accelerate

Improve the performance of your Java application by using these optimizations

Understanding Spark Re-Partition

Improve the performance of your Java application by using these optimizations

#memory-management
#memory-management