📍성능 개선을 하게된 계기
이 당시 기능 구현에만 집중해서 코드를 구현했기 때문에 성능을 고려하지 못했다.
지인의 블로그를 읽는 중, 성능 관련 테스트 방법이 보이길래 나도 진행해 보게 되었다.
처참하게 많이 발생하는 쿼리 수를 보고 쿼리 개선을 해야겠다는 생각과 함께 성능 개선 시작..
(K6 툴 사용)
-> 지인 블로그
https://velog.io/@hyeok_1212/%EC%A1%B0%ED%9A%8C%EC%88%98-%EC%A0%95%ED%95%A9%EC%84%B1
조회수 기능 구현 (동시성 이슈)
조회수는 사용자가 콘텐츠에 대한 관심을 가장 직관적으로 확인할 수 있는 지표에요. 서비스의 품질과 사용자 경험에 긍정적인 영향을 주는 기능이지만, 단순한 구현은 신뢰도를 떨어뜨릴 위험
velog.io
📍기존 코드 분석
@Override
public Page<RecommendedFollowInfoResDto> findRecommendedFollowList(Long memberId, Pageable pageable) {
List<Member> potentialFriends = queryFactory
.select(member)
.from(teamDashboardMemberMapping)
.join(teamDashboardMemberMapping.member, member)
.join(teamDashboardMemberMapping.teamDashboard, teamDashboard)
.where(
teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboard)
.where(
teamDashboard.member.id.eq(memberId)
.or(teamDashboardMemberMapping.member.id.eq(memberId))
)
)
)
.where(member.id.ne(memberId))
.fetch();
List<Member> dashboardOwners = queryFactory
.select(teamDashboard.member)
.from(teamDashboard)
.where(teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboardMemberMapping)
.where(teamDashboardMemberMapping.member.id.eq(memberId))
))
.fetch();
potentialFriends.addAll(dashboardOwners);
potentialFriends = potentialFriends.stream().distinct().collect(Collectors.toList());
List<RecommendedFollowInfoResDto> recommendedFollows = potentialFriends.stream()
.filter(teamMember -> !teamMember.getId().equals(memberId)) // 본인 제외
.filter(teamMember -> {
boolean isFollow = queryFactory
.selectOne()
.from(follow)
.where(
(follow.fromMember.id.eq(memberId).and(follow.toMember.id.eq(teamMember.getId())))
.or(follow.fromMember.id.eq(teamMember.getId())
.and(follow.toMember.id.eq(memberId)))
)
.fetchFirst() != null;
return !isFollow;
})
.map(teamMember -> RecommendedFollowInfoResDto.from(teamMember, false))
.collect(Collectors.toList());
int start = (int) pageable.getOffset();
int end = Math.min((start + pageable.getPageSize()), recommendedFollows.size());
List<RecommendedFollowInfoResDto> pagedRecommendedFollows = recommendedFollows.subList(start, end);
return new PageImpl<>(pagedRecommendedFollows, pageable, recommendedFollows.size());
}쿼리 내용을 한 줄씩 분석해보자.
1. 같은 대시보드에 속한 멤버 + 현재 회원이 참여한 대시보드의 소유자 + 중복 제거
2. 이미 팔로우한 멤버 제외 + 페이징 처리
로직은 맞지만 전체적으로 많은 쿼리가 발생. 실제로 모든 멤버의 follow 정보를 하나씩 확인하기에 데이터가 많아질수록 상당한 시간이 소요된다.
team-dashboard-memeber-mapping안에 튜플이 0일 때
data_received..................: 4.0 MB 66 kB/s
data_sent......................: 5.3 MB 88 kB/s
http_req_blocked...............: avg=14.4µs min=1µs med=6µs max=3.9ms p(90)=11µs p(95)=24µs
http_req_connecting............: avg=4.42µs min=0s med=0s max=3.71ms p(90)=0s p(95)=0s
http_req_duration..............: avg=43.57ms min=8.26ms med=29.99ms max=962.04ms p(90)=83.53ms p(95)=112.98ms
{ expected_response:true }...: avg=43.57ms min=8.26ms med=29.99ms max=962.04ms p(90)=83.53ms p(95)=112.98ms
http_req_failed................: 0.00% 0 out of 13657
http_req_receiving.............: avg=869.1µs min=22µs med=293µs max=688.24ms p(90)=1.02ms p(95)=2.05ms
http_req_sending...............: avg=29.66µs min=5µs med=21µs max=2.88ms p(90)=50µs p(95)=71µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=42.67ms min=8.04ms med=29.51ms max=961.88ms p(90)=81.48ms p(95)=111.28ms
http_reqs......................: 13657 227.541426/s
iteration_duration.............: avg=43.9ms min=8.57ms med=30.33ms max=962.38ms p(90)=83.89ms p(95)=113.37ms
iterations.....................: 13657 227.541426/s
vus............................: 10 min=10 max=10
vus_max........................: 10 min=10 max=10
team-dashboard-memeber-mapping안에 튜플이 10000개일 때
data_received..................: 31 kB 381 B/s
data_sent......................: 7.7 kB 95 B/s
http_req_blocked...............: avg=621.9µs min=2µs med=339.5µs max=1.41ms p(90)=1.38ms p(95)=1.4ms
http_req_connecting............: avg=193.7µs min=0s med=153µs max=471µs p(90)=418.1µs p(95)=430.15µs
http_req_duration..............: avg=40.58s min=36.66s med=40.46s max=44.64s p(90)=44.61s p(95)=44.64s
{ expected_response:true }...: avg=40.58s min=36.66s med=40.46s max=44.64s p(90)=44.61s p(95)=44.64s
http_req_failed................: 0.00% 0 out of 20
http_req_receiving.............: avg=14.84ms min=615µs med=20.2ms max=42.45ms p(90)=24.43ms p(95)=25.62ms
http_req_sending...............: avg=295.84µs min=5µs med=280.5µs max=645µs p(90)=613.19µs p(95)=625.04µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=40.56s min=36.64s med=40.44s max=44.64s p(90)=44.61s p(95)=44.64s
http_reqs......................: 20 0.245921/s
iteration_duration.............: avg=40.58s min=36.67s med=40.46s max=44.64s p(90)=44.61s p(95)=44.64s
iterations.....................: 20 0.245921/s
vus............................: 9 min=9 max=10
vus_max........................: 10 min=10 max=10- 튜플 증가 시 응답 시간이 43ms에서 40s로 1000배 이상 증가한 것이 확인된다. N+1 발생이 난무하는 쿼리..
📍N+1 문제 해결
원인 분석
.filter(teamMember -> {
boolean isFollow = queryFactory
.selectOne()
.from(follow)
.where(
(follow.fromMember.id.eq(memberId).and(follow.toMember.id.eq(teamMember.getId())))
.or(follow.fromMember.id.eq(teamMember.getId())
.and(follow.toMember.id.eq(memberId)))
)
.fetchFirst() != null;
return !isFollow;
})isFollow를 확인하기 위한 호출이 team-dashboard-memeber-mapping 테이블의 튜플 개수에 비례해서 일어나고 있다.
@Override
public Page<RecommendedFollowInfoResDto> findRecommendedFollowList(Long memberId, Pageable pageable) {
List<Member> potentialFriends = queryFactory
.select(member)
.from(teamDashboardMemberMapping)
.join(teamDashboardMemberMapping.member, member)
.join(teamDashboardMemberMapping.teamDashboard, teamDashboard)
.where(
teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboard)
.where(
teamDashboard.member.id.eq(memberId)
.or(teamDashboardMemberMapping.member.id.eq(memberId))
)
)
)
.where(member.id.ne(memberId))
.fetch();
List<Member> dashboardOwners = queryFactory
.select(teamDashboard.member)
.from(teamDashboard)
.where(teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboardMemberMapping)
.where(teamDashboardMemberMapping.member.id.eq(memberId))
))
.fetch();
potentialFriends.addAll(dashboardOwners);
potentialFriends = potentialFriends.stream().distinct().collect(Collectors.toList());
// 팔로우 상태를 한 번의 배치 쿼리로 조회
Map<Long, Boolean> followStatusMap = queryFactory
.select(follow.toMember.id, follow.fromMember.id)
.from(follow)
.where(
follow.fromMember.id.eq(memberId)
.or(follow.toMember.id.eq(memberId))
)
.fetch()
.stream()
.collect(Collectors.toMap(
result -> result.get(follow.toMember.id),
result -> true
));
List<RecommendedFollowInfoResDto> recommendedFollows = potentialFriends.stream()
.filter(teamMember -> !followStatusMap.containsKey(teamMember.getId()))
.map(teamMember -> RecommendedFollowInfoResDto.from(teamMember, false))
.collect(Collectors.toList());
int start = (int) pageable.getOffset();
int end = Math.min((start + pageable.getPageSize()), recommendedFollows.size());
List<RecommendedFollowInfoResDto> pagedRecommendedFollows = recommendedFollows.subList(start, end);
return new PageImpl<>(pagedRecommendedFollows, pageable, recommendedFollows.size());
}이렇게 한 번의 배치 쿼리로 조회해서 N+1문제를 해결하면
data_received..................: 458 kB 7.4 kB/s
data_sent......................: 115 kB 1.9 kB/s
http_req_blocked...............: avg=116.88µs min=2µs med=7µs max=3.91ms p(90)=14µs p(95)=34.74µs
http_req_connecting............: avg=17.79µs min=0s med=0s max=976µs p(90)=0s p(95)=0s
http_req_duration..............: avg=2.06s min=736.28ms med=1.86s max=7.03s p(90)=3.03s p(95)=3.91s
{ expected_response:true }...: avg=2.06s min=736.28ms med=1.86s max=7.03s p(90)=3.03s p(95)=3.91s
http_req_failed................: 0.00% 0 out of 296
http_req_receiving.............: avg=5.35ms min=104µs med=876µs max=183.58ms p(90)=7.65ms p(95)=17.5ms
http_req_sending...............: avg=31.92µs min=5µs med=23µs max=478µs p(90)=39µs p(95)=62.24µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=2.06s min=734.76ms med=1.85s max=6.84s p(90)=3.03s p(95)=3.86s
http_reqs......................: 296 4.792305/s
iteration_duration.............: avg=2.06s min=736.62ms med=1.86s max=7.03s p(90)=3.03s p(95)=3.91s
iterations.....................: 296 4.792305/s
vus............................: 9 min=9 max=10
vus_max........................: 10 min=10 max=10응답 시간이 기존 40초에서 2초로 95% 단축된 것을 볼 수 있다.
📍페이징 로직 추가
지금 로직은 모든 데이터를 가져와서 처리하고 있다.
Offset과 PageSize를 따로 받는 상황이기 때문에, 굳이 이럴 필요 없이 딱 요청 사항만 가져오도록 설계해보자.
@Override
public Page<RecommendedFollowInfoResDto> findRecommendedFollowList(Long memberId, Pageable pageable) {
List<Member> potentialFriends = queryFactory
.select(member)
.from(teamDashboardMemberMapping)
.join(teamDashboardMemberMapping.member, member)
.join(teamDashboardMemberMapping.teamDashboard, teamDashboard)
.where(
teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboard)
.where(
teamDashboard.member.id.eq(memberId)
.or(teamDashboardMemberMapping.member.id.eq(memberId))
)
)
)
.where(member.id.ne(memberId))
.offset(pageable.getOffset()) // 페이징 로직 추가. 딱 요청 사항만 가져옴.
.limit(pageable.getPageSize())
.fetch();
List<Member> dashboardOwners = queryFactory
.select(teamDashboard.member)
.from(teamDashboard)
.where(teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboardMemberMapping)
.where(teamDashboardMemberMapping.member.id.eq(memberId))
))
.fetch();
potentialFriends.addAll(dashboardOwners);
potentialFriends = potentialFriends.stream().distinct().collect(Collectors.toList());
Map<Long, Boolean> followStatusMap = queryFactory
.select(follow.toMember.id, follow.fromMember.id)
.from(follow)
.where(
follow.fromMember.id.eq(memberId)
.or(follow.toMember.id.eq(memberId))
)
.fetch()
.stream()
.collect(Collectors.toMap(
result -> result.get(follow.toMember.id),
result -> true
));
List<RecommendedFollowInfoResDto> recommendedFollows = potentialFriends.stream()
.filter(teamMember -> !followStatusMap.containsKey(teamMember.getId()))
.map(teamMember -> RecommendedFollowInfoResDto.from(teamMember, false))
.collect(Collectors.toList());
int start = (int) pageable.getOffset();
int end = Math.min((start + pageable.getPageSize()), recommendedFollows.size());
List<RecommendedFollowInfoResDto> pagedRecommendedFollows = recommendedFollows.subList(start, end);
return new PageImpl<>(pagedRecommendedFollows, pageable, recommendedFollows.size());
} data_received..................: 11 MB 177 kB/s
data_sent......................: 2.7 MB 45 kB/s
http_req_blocked...............: avg=14.64µs min=1µs med=7µs max=1.73ms p(90)=10µs p(95)=22µs
http_req_connecting............: avg=3.78µs min=0s med=0s max=1.59ms p(90)=0s p(95)=0s
http_req_duration..............: avg=86.36ms min=18.73ms med=71.46ms max=2.12s p(90)=145.21ms p(95)=180.93ms
{ expected_response:true }...: avg=86.36ms min=18.73ms med=71.46ms max=2.12s p(90)=145.21ms p(95)=180.93ms
http_req_failed................: 0.00% 0 out of 6906
http_req_receiving.............: avg=679.42µs min=12µs med=285µs max=51.87ms p(90)=851µs p(95)=1.54ms
http_req_sending...............: avg=31.86µs min=4µs med=23µs max=1.86ms p(90)=49µs p(95)=66µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=85.65ms min=18.61ms med=70.96ms max=2.11s p(90)=143.5ms p(95)=178.84ms
http_reqs......................: 6906 115.003411/s
iteration_duration.............: avg=86.89ms min=18.95ms med=72.1ms max=2.13s p(90)=145.88ms p(95)=181.44ms
iterations.....................: 6906 115.003411/s
vus............................: 10 min=10 max=10
vus_max........................: 10 min=10 max=10wow 2초에서 86.36ms로 약 24배 단축된 것을 확인할 수 있다!
📍중복 데이터를 메모리 딴에서 제거
potentialFriends와 dashboardOwners를 가져온 후, Java 스트림에서 중복을 제거하고 있다.
중복을 데이터베이스에서 제거하지 않고, 메모리로 데이터를 가져와 처리하면 데이터가 많을 때 성능 문제가 발생할 수 있기 때문에 데이터베이스에서 바로 제거하도록 하자.
@Override
public Page<RecommendedFollowInfoResDto> findRecommendedFollowList(Long memberId, Pageable pageable) {
List<Long> excludedIds = queryFactory
.select(follow.fromMember.id)
.from(follow)
.where(
follow.toMember.id.eq(memberId)
.or(follow.fromMember.id.eq(memberId))
.or(
follow.fromMember.id.in(
queryFactory
.select(follow.toMember.id)
.from(follow)
.where(follow.fromMember.id.eq(memberId))
)
)
.or(
follow.toMember.id.in(
queryFactory
.select(follow.fromMember.id)
.from(follow)
.where(follow.toMember.id.eq(memberId))
)
)
)
.fetch();
List<Member> potentialFriends = queryFactory
.select(member)
.distinct()
.from(teamDashboardMemberMapping)
.join(teamDashboardMemberMapping.member, member)
.join(teamDashboardMemberMapping.teamDashboard, teamDashboard)
.where(
teamDashboard.id.in(
queryFactory
.select(teamDashboard.id)
.from(teamDashboard)
.where(
teamDashboard.member.id.eq(memberId)
.or(teamDashboardMemberMapping.member.id.eq(memberId))
)
)
.or(teamDashboardMemberMapping.teamDashboard.id.in(
queryFactory
.select(teamDashboardMemberMapping.teamDashboard.id)
.from(teamDashboardMemberMapping)
.where(teamDashboardMemberMapping.member.id.eq(memberId))
))
)
.where(member.id.ne(memberId))
.where(member.id.notIn(excludedIds))
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
.fetch();
List<RecommendedFollowInfoResDto> recommendedFollows = potentialFriends.stream()
.map(teamMember -> RecommendedFollowInfoResDto.from(teamMember, false))
.collect(Collectors.toList());
return new PageImpl<>(recommendedFollows, pageable, recommendedFollows.size());
}두 리스트를 별도로 가져오지 않고, 쿼리 하나로 통합하여 중복 로직을 제거했다.
data_received..................: 9.6 MB 160 kB/s
data_sent......................: 2.4 MB 40 kB/s
http_req_blocked...............: avg=19.41µs min=1µs med=7µs max=3.9ms p(90)=11µs p(95)=20µs
http_req_connecting............: avg=4.68µs min=0s med=0s max=1.26ms p(90)=0s p(95)=0s
http_req_duration..............: avg=95.76ms min=19.3ms med=77.39ms max=2.32s p(90)=159.76ms p(95)=199.31ms
{ expected_response:true }...: avg=95.76ms min=19.3ms med=77.39ms max=2.32s p(90)=159.76ms p(95)=199.31ms
http_req_failed................: 0.00% 0 out of 6233
http_req_receiving.............: avg=743.4µs min=22µs med=292µs max=67.16ms p(90)=907µs p(95)=1.79ms
http_req_sending...............: avg=32.53µs min=5µs med=23µs max=6.78ms p(90)=47µs p(95)=65µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=94.99ms min=18.84ms med=76.82ms max=2.27s p(90)=158.45ms p(95)=198.11ms
http_reqs......................: 6233 103.710728/s
iteration_duration.............: avg=96.34ms min=19.59ms med=77.92ms max=2.33s p(90)=160.38ms p(95)=199.99ms
iterations.....................: 6233 103.710728/s
vus............................: 10 min=10 max=10
vus_max........................: 10 min=10 max=10그렇게 결과를 분석해 보았는데 응답 시간은 비슷한데 데이터 전송량과 요청 처리량이 줄어들었다. 왤까?
-> 불필요한 데이터가 줄어들었기 때문에 데이터 전송량이 감소.
-> 페이징 조건 등으로 인해 필요한 데이터만 처리했기 때문에 요청 처리량이 감소.
📍결론
쿼리 개선으로 인해서 어느정도 성능을 향상시킨 것 같다.
다만 내가 짠 쿼리에 가독성이 좋아보이진 않는다.
더 좋은 쿼리가 있을지 계속해서 고민해보자.
'springboot' 카테고리의 다른 글
| [SpringBoot] N+1 문제와 해결 방법 (0) | 2025.01.10 |
|---|---|
| [SpringBoot] 영속성 컨텍스트, 그리고 EntityManager (0) | 2025.01.09 |
| [SpringBoot] QueryDSL 그게 뭔데 다들 쓰는거지? (0) | 2024.12.28 |
| [SpringBoot] 웹소켓으로 채팅하고 채팅방 관리하기 (0) | 2024.11.23 |
| [SpringBoot] Session을 활용한 회원가입/로그인 유지 (1) | 2024.11.01 |