设为首页 加入收藏

TOP

leveldb:dbimpl(1)(二)
2015-11-21 01:43:20 来源: 作者: 【 】 浏览:1
Tags:leveldb:dbimpl
e定级,不能和同级的sstable有overlap,也不能和上级的sstable overlap太多(> kMaxGrandParentOverlapBytes)
WriteLevel0Table是函数CompactMemTable的核心。

leveldb中有且只有一个进程单独做compact,当主线程触发compact,调用void DBImpl::MaybeScheduleCompaction(),如果compact正在运行或者DB正在退出,直接返回;检查version中是否存在需要compact,有则触发后台调度env_->schedele(…)

void DBImpl::MaybeScheduleCompaction() { mutex_.AssertHeld(); if (bg_compaction_scheduled_) { // Already scheduled } else if (shutting_down_.Acquire_Load()) { // DB is being deleted; no more background compactions } else if (!bg_error_.ok()) { // Already got an error; no more changes } else if (imm_ == NULL && manual_compaction_ == NULL && !versions_->NeedsCompaction()) { // No work to be done } else { bg_compaction_scheduled_ = true; env_->Schedule(&DBImpl::BGWork, this); } }

schedele把compact处理程序函数指针和db对象指针传入后台任务队列,BGWork 是compact处理函数,Schedule函数如下:

void PosixEnv::Schedule(void (*function)(void*), void* arg) { PthreadCall("lock", pthread_mutex_lock(&mu_)); // Start background thread if necessary if (!started_bgthread_) { started_bgthread_ = true; PthreadCall( "create thread", pthread_create(&bgthread_, NULL, &PosixEnv::BGThreadWrapper, this)); } // If the queue is currently empty, the background thread may currently be // waiting. if (queue_.empty()) { PthreadCall("signal", pthread_cond_signal(&bgsignal_)); } // Add to priority queue queue_.push_back(BGItem()); queue_.back().function = function; queue_.back().arg = arg; PthreadCall("unlock", pthread_mutex_unlock(&mu_)); }

将处理函数放入任务队列中,后台进程就可以不断地从queue_中取出任务函数,并执行。

实际compact处理进程是BackgroundCall和BackgroundCompaction。BackgroundCall完成一些判断,条件符合则调用BackgroundCompaction,compact完成后再次触发compact,重复上述过程。

void DBImpl::BackgroundCall() { MutexLock l(&mutex_); assert(bg_compaction_scheduled_); if (shutting_down_.Acquire_Load()) { // No more background work when shutting down. } else if (!bg_error_.ok()) { // No more background work after a background error. } else { BackgroundCompaction(); } bg_compaction_scheduled_ = false; // Previous compaction may have produced too many files in a level, // so reschedule another compaction if needed. MaybeScheduleCompaction(); bg_cv_.SignalAll(); }

实际compact流程:

void DBImpl::BackgroundCompaction() { mutex_.AssertHeld(); //immutable先compact if (imm_ != NULL) { CompactMemTable(); return; } //针对人为指定compact的key-range Compaction* c; bool is_manual = (manual_compaction_ != NULL); InternalKey manual_end; if (is_manual) { ManualCompaction* m = manual_compaction_; c = versions_->CompactRange(m->level, m->begin, m->end); m->done = (c == NULL); if (c != NULL) { manual_end = c->input(0, c->num_input_files(0) - 1)->largest; } Log(options_.info_log, "Manual compaction at level-%d from %s .. %s; will stop at %s\n", m->level, (m->begin ? m->begin->DebugString().c_str() : "(begin)"), (m->end ? m->end->DebugString().c_str() : "(end)"), (m->done ? "(end)" : manual_end.DebugString().c_str())); } else { //确定需要compact的level-n和sstable c = versions_->PickCompaction(); } Status status; if (c == NULL) { // Nothing to do } else if (!is_manual && c->IsTrivialMove()) { // Move file to next level assert(c->num_input_files(0) == 1); FileMetaData* f = c->input(0, 0); c->edit()->DeleteFile(c->level(), f->number); c->edit()->AddFile(c->level() + 1, f->number, f->file_size, f->smallest, f->largest); status = versions_->LogAndApply(c->edit(), &mutex_
首页 上一页 1 2 3 4 5 下一页 尾页 2/5/5
】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
分享到: 
上一篇leveldb学习:Env 下一篇SqlServer数据库还原(请勿模仿!..

评论

帐  号: 密码: (新用户注册)
验 证 码:
表  情:
内  容: